System and method for codec for combining disparate content

ABSTRACT

A method for encoding and decoding disparate content includes receiving, at a storage location, a combined file created by an encoder, wherein the combined file includes a plurality of data of one or more content types, receiving, from a client device, a request for retrieval of the combined file, determining a compatibility of the client device to decode and view content of the combined file, and transmitting, based on the determination, the combined file to a decoder, wherein the decoder separates the combined file into the plurality of data of the one or more content types.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/934,183, filed Jul. 21, 2020 (Atty. Dkt. No. JCHR60-34642), which isa continuation-in-part of U.S. patent application Ser. No. 16/251,581,filed Jan. 18, 2019, and entitled SYSTEM AND METHOD FOR ENCODING IMAGEDATA AND OTHER DATA TYPES INTO ONE DATA FORMAT AND DECODING OF SAME.U.S. patent application Ser. No. 16/251,581 is a continuation of U.S.patent application Ser. No. 15/785,148, filed Oct. 16, 2017, whichissued as U.S. Pat. No. 10,187,443 on Jan. 22, 2019, and entitled SYSTEMAND METHOD FOR ENCODING IMAGE DATA AND OTHER DATA TYPES INTO ONE DATAFORMAT AND DECODING OF SAME. U.S. patent application Ser. No. 15/785,148claims priority to and the benefit of U.S. Provisional PatentApplication No. 62/518,034, filed Jun. 12, 2017, and entitled DATAFORMAT SPECIFICATION AND METHOD OF BLENDING AND SEPARATING IMAGE DATASTREAM OR FILES AND OTHER DATA STREAM OR FILES INTO AND FROM THE DATAFORMAT STREAM OR FILE. This application also claims priority to and thebenefit of U.S. Provisional Patent Application No. 62/876,940, filedJul. 22, 2019, and entitled SYSTEM AND METHOD FOR CONTINUOUS ENRICHMENTOF ENCODED CONTENT AND SECURITY MEASURE FOR DECODING OF SAME. Thisapplication also claims priority to and the benefit of U.S. ProvisionalPatent Application No. 62/988,030, filed Mar. 11, 2020, and entitledSYSTEM AND METHOD FOR DATA MOBILITY AND SCARCITY OF ENCODED CONTENT.This application also claims priority to and the benefit of U.S.Provisional Patent Application No. 63/002,495, filed Mar. 31, 2020, andentitled SYSTEM AND METHOD FOR SELECTIVE ENCRYPTION OF ENCODED CONTENT.The contents of U.S. Pat. No. 10,187,443 and U.S. patent applicationSer. Nos. 16/251,581, 15/785,148, 62/518,034, 62/876,940, 62/988,030,and 63/002,495 are incorporated by reference herein in their entirety.

TECHNICAL FIELD

The present disclosure is related generally to data exchange andstorage. More particularly the present disclosure relates to theblending of an image data stream or file and another non-image datastream or file into a single data format.

BACKGROUND

Image data streaming or files are popular media to share pictures oncomputers or devices, over the internet or other networks, and areviewed on a number of different computing devices, like smartphones. Butin many cases, it may be desirable to share or store other informationwhile viewing and storing the image data, especially to see the pictureand hear audio at the same time.

Current techniques for this usage include storing or exchanging theimage data or file separately from the non-image data or file. Forexample, users store or exchange the pictures with JPEG files, and thevoice or audio data in way or mp3 files alongside the image files. Theproblem with this method is that, in order to see the image and hear theassociated audio at the same time, the users have to do two actions tosave or transfer two data files, one for the image, another for theaudio.

Video media data or file has both “moving” image and audio informationdata, but this is a different media usage. The video takes more space tostore, and wider network bandwidth to exchange, and the resolution ofthe image frames in the video file is much lower than the resolutionthat can make up the image data format.

Therefore, what is needed is a method and technique for blending theimage data stream or file along with non-image data or files in a singledata stream or file, a method and technique for separating and returningthe image data stream or file and the non-image streams or files fromthe single blended data stream or file generated, a stream data or filedata structure or format for the blended single stream or file, and anapplication program to implement the methods of blending and separating,noted above.

SUMMARY

In one aspect thereof, a method of a codec for encoding data streamsinto a combined file is provided. The method comprises accessing a firstfile having a first plurality of data bytes, accessing a second filehaving a second plurality of data bytes, combining the first file andthe second file to provide the combined file containing a header and abody, comprising the steps of in a first storing step, storing a blockof data bytes of a first byte block size, from the first plurality ofdata bytes, in the body of the combined file as a first file byte block,wherein a byte block size includes at least one or more bytes of data,in a second storing step, sequentially storing a block of data bytes ofa second byte block size, from the second plurality of data bytes, inthe body of the combined file as a second file byte block, repeating thefirst and second storing steps to sequentially store all of the databytes in the first file and the second file in the combined file, andstoring, in the header, information relating to the first byte blocksize and the second byte block size.

In another aspect thereof, a system for encoding data streams into acombined file and decoding the combined file into separate data streamsis provided. The system comprises a network interface coupled to aprocessor and a memory to the processor, the processor configured toaccess a first file having a first plurality of data bytes, access asecond file having a second plurality of data bytes, combine the firstfile and the second file to provide the combined file containing aheader and a body, wherein, during combining, the processor is furtherconfigured to in a first storing step, store a block of data bytes of afirst byte block size, from the first plurality of data bytes, in thebody of the combined file as a first file byte block, wherein a byteblock size includes at least one or more bytes of data, in a secondstoring step, sequentially store a block of data bytes of a second byteblock size, from the second plurality of data bytes, in the body of thecombined file as a second file byte block, repeat the first and secondstoring steps to sequentially store all of the data bytes in the firstfile and the second file in the combined file, and store, in the header,information relating to the first byte block size and the second byteblock size.

A data structure or format (hereinafter referred to as chm format, chifformat, CHM, or CHIF) for the blended stream or file is created inaccordance with this disclosure. The data structure or format has twoparts: the metadata bytes at the header section in the beginning and theraw data bytes at the body section. Along with the chm data format, aprotocol (hereinafter referred to as chm formatting) is created to blendan image data stream or file with one or more non-image data streams orfiles.

In one aspect, a method of blending an image file with a non-image fileis provided. The method may begin with accessing both image andnon-image data streams or files by the application program whichimplements the technology set forth by this disclosure. Once accessed,the application may read the data information of both image andnon-image streams or files, and based on the chm format, it may createand write the metadata bytes into the beginning header section of a chmformat data stream or file. Next, the application may read one chunk ofdata bytes from each of the image and non-image data streams or files,and write the two chunks of data bytes into the chm format stream orfile. And the application may continue and repeat reading one chunk ofdata bytes from two data streams and writing the two chunks of databytes into the chm format data stream or file until it reaches the endof the two image and non-image data streams or files. The process ofthis method is called “chm encoding.”

In another aspect, a method of separating the chm format data stream orfile and returning back the image stream or file and the non-imagestream or file is provided. The method may begin with accessing the chmformat data stream or file which is generated by the blending methodabove with the application program. Once accessed, the application mayread and retrieve the metadata information from a header section of thestream or file. Next, based on protocol, the application may read twochunks of bytes from the chm format data stream or file, and it maywrite one chunk of bytes into the image stream or file, and anotherchunk of bytes into the non-image stream or file. And the applicationmay continue and repeat reading the next two chunks of bytes from thechm format data stream, and writing each chunk of the bytes into theirown data streams or files until it reaches the end of the chm formatdata stream or file, and it returns the image and non-image data streamsor files back to their original states. The process of this method iscalled “chm decoding.”

The application program introduced above is the software whichimplements the blending/separating methods and the protocol to executethe processes to blend/separate the image the non-image data streams orfiles into/from the single chm format data stream or file.

In another aspect, a method for encoding and decoding disparate contentincludes receiving, at a storage location, a combined file created by anencoder, wherein the combined file includes a plurality of data of oneor more content types. The method further includes receiving, from aclient device, a request for retrieval of the combined file. The methodfurther includes determining a compatibility of the client device todecode and view content of the combined file. The method furtherincludes transmitting, based on the determination, the combined file toa decoder, wherein the decoder separates the combined file into theplurality of data of the one or more content types.

In another aspect, an electronic device includes a memory, a networkinterface, and at least one processor coupled to the memory and thenetwork interface. The at least one processor is configured to receive acombined file created by an encoder, wherein the combined file includesa plurality of data of one or more content types. The at least oneprocessor is further configured to receive, from a client device, arequest for retrieval of the combined file. The at least one processoris further configured to determine a compatibility of the client deviceto decode and view content of the combined file. The at least oneprocessor is further configured to transmit, via the network interface,and based on the determination, the combined file to a decoder, whereinthe decoder separates the combined file into the plurality of data ofthe one or more content types.

In another aspect, an electronic device includes a memory, a networkinterface, and at least one processor coupled to the memory and thenetwork interface. The at least one processor is configured to receive,from a client device, a request for decryption and decoding of acombined file, wherein the combined file includes metadata and pluralityof data of one or more content types, wherein the metadata includes auniversally unique identifier (UUID), and wherein the combined file isencrypted. The at least one processor is further configured to decryptthe combined file. The at least one processor is further configured todetermine, based on the UUID, that decoding of the combined file isallowed. The at least one processor is further configured to decode,based on the determination, the combined file, wherein the decodingseparates the combined file into the plurality of data of the one ormore content types. The at least one processor is further configured totransmit, via the network interface, the plurality of data of the one ormore content types to the client device.

In another aspect, an electronic device includes a memory and at leastone processor coupled to the memory. The at least one processor isconfigured to receive a combined file created by an encoder, wherein thecombined file includes metadata and an image associated with a pluralityof data of one or more content types. The at least one processor isfurther configured to feed at least one of the plurality of data into atleast one artificial intelligence model. The at least one processor isfurther configured to receive one or more outputs from the at least oneartificial intelligence model. The at least one processor is furtherconfigured to create an enriched combined file by modifying the metadataof the combined file based on at least a portion of the one or moreoutputs from the at least one artificial intelligence model, The atleast one processor is further configured to perform on the enrichedcombined file at least one of: analytics, indexing, or objectrecognition.

In another aspect, a method of an electronic device for selectiveencryption of content to be encoded into a container includes receiving,by a processor of the electronic device, a plurality of content itemsfor encoding into a container, wherein each one of the plurality ofcontent items is associated with a content key value, receiving, by theprocessor, one or more content selections from among the plurality ofcontent items, retrieving, by the processor, encryption data for use inencrypting the one or more content selections, encrypting, by theprocessor, at least one content selection of the one or more contentselections using an encryption key derived using the encryption data,for the encrypted at least one content selection, associating, by theprocessor, a corresponding content key value of the encrypted at leastone content selection with the encryption data, encoding, by theprocessor, the associated corresponding content key value of theencrypted at least one content selection and the encryption data intothe container, and encoding, by the processor, the encrypted at leastone content selection and remaining content items of the plurality ofcontent items into the container.

In another aspect, an electronic device includes a memory, and at leastone processor coupled to the memory. The at least one processor isconfigured to receive a plurality of content items for encoding into acontainer, wherein each one of the plurality of content items isassociated with a content key value, receive one or more contentselections from among the plurality of content items, retrieveencryption data for use in encrypting the one or more contentselections, encrypt at least one content selection of the one or morecontent selections using an encryption key derived using the encryptiondata, for the encrypted at least one content selection, associate acorresponding content key value of the encrypted at least one contentselection with the encryption data, encode the associated correspondingcontent key value of the encrypted at least one content selection andthe encryption data into the container, and encode the encrypted atleast one content selection and remaining content items of the pluralityof content items into the container.

In another aspect, a method for encoding and decoding disparate contentincludes receiving, at a storage location, a combined file created by anencoder, wherein the combined file includes a plurality of data of oneor more content types, receiving, from a client device, a request forretrieval of the combined file, determining a compatibility of theclient device to decode and view content of the combined file, andtransmitting, based on the determination, the combined file to adecoder, wherein the decoder separates the combined file into theplurality of data of the one or more content types.

In another aspect, an electronic device includes a memory, a networkinterface, and at least one processor coupled to the memory and thenetwork interface. The at least one processor is configured to receive acombined file created by an encoder, wherein the combined file includesa plurality of data of one or more content types, receive, from a clientdevice, a request for retrieval of the combined file, determine acompatibility of the client device to decode and view content of thecombined file, and transmit, via the network interface, and based on thedetermination, the combined file to a decoder, wherein the decoderseparates the combined file into the plurality of data of the one ormore content types.

In another aspect, an electronic device includes a memory, a networkinterface, and at least one processor coupled to the memory and thenetwork interface. The at least one processor is configured to receive,from a client device, a request for decryption and decoding of acombined file, wherein the combined file includes metadata and pluralityof data of one or more content types, wherein the metadata includes auniversally unique identifier (UUID), and wherein the combined file isencrypted, decrypt the combined file, determine, based on the UUID, thatdecoding of the combined file is allowed, decode, based on thedetermination, the combined file, wherein the decoding separates thecombined file into the plurality of data of the one or more contenttypes, and transmit, via the network interface, the plurality of data ofthe one or more content types to the client device.

In another aspect, an electronic device includes a memory, and at leastone processor coupled to the memory. The at least one processor isconfigured to receive a combined file created by an encoder, wherein thecombined file includes metadata and an image associated with a pluralityof data of one or more content types, feed at least one of the pluralityof data into at least one artificial intelligence model, receive one ormore outputs from the at least one artificial intelligence model, createan enriched combined file by modifying the metadata of the combined filebased on at least a portion of the one or more outputs from the at leastone artificial intelligence model, and perform on the enriched combinedfile at least one of: analytics, indexing, or object recognition.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding, reference is now made to thefollowing description taken in conjunction with the accompanyingDrawings in which:

FIG. 1 illustrates a flowchart of a CHM encoding and decoding processfor image and non-image data streams, in accordance with variousembodiments of the present disclosure;

FIG. 2 illustrates a data structure or format, referred to as CHMformat, for a blended stream or file, in accordance with variousembodiments of the present disclosure;

FIG. 3A illustrates a flowchart of a method of blending or encoding animage stream or file with a non-image stream or file into a CHM formatdata stream, in accordance with various embodiments of the presentdisclosure;

FIG. 3B illustrates a flowchart of a method of separating or decoding animage stream or file and a non-image stream or file from a CHM formatdata stream, in accordance with various embodiments of the presentdisclosure;

FIG. 4 illustrates a diagrammatic view of one embodiment of a CHM fileencoding process, in accordance with various embodiments of the presentdisclosure;

FIG. 5 illustrates a flowchart of a CHM creation process in accordancewith various embodiments of the present disclosure;

FIG. 6 illustrates a diagrammatic view of one embodiment of a CHM filedecoding process in accordance with various embodiments of the presentdisclosure;

FIG. 7 illustrates a CHM decoding process in accordance with variousembodiments of the present disclosure;

FIG. 8 illustrates a diagrammatic view of an embodiment of a CHM fileencoding process where the files to be blended into a CHM file do nothave an equal number of bytes, in accordance with various embodiments ofthe present disclosure;

FIGS. 9A and 9B illustrate a flowchart of a CHM encoding process inaccordance with various embodiments of the present disclosure;

FIG. 10 illustrates a diagrammatic view of an embodiment of a CHM filedecoding process where the files to be decoded from a CHM file do nothave an equal number of bytes, in accordance with various embodiments ofthe present disclosure;

FIG. 11 illustrates a flowchart of a CHM decoding process in accordancewith various embodiments of the present disclosure;

FIG. 12 illustrates a diagrammatic view of a server-side CHM filedecoding and transmission system in accordance with various embodimentsof the present disclosure;

FIG. 13 illustrates a browser window showing a webpage including animage with accompanying audio presented on the webpage, in accordancewith various embodiments of the present disclosure;

FIG. 14 illustrates a flowchart of a server-side CHM decoding process inaccordance with various embodiments of the present disclosure;

FIG. 15 illustrates a medical imaging and dictation CHM file creationsystem in accordance with various embodiments of the present disclosure;

FIG. 16 illustrates a medical imaging and dictation CHM file creationprocess in accordance with various embodiments of the presentdisclosure;

FIG. 17 illustrates a diagrammatic view of a CHM binder file creationprocess in accordance with various embodiments of the presentdisclosure;

FIG. 18 illustrates a diagrammatic view of a CHM binder file decodingprocess in accordance with various embodiments of the presentdisclosure;

FIG. 19 illustrates a binder file creation and decoding process inaccordance with various embodiments of the present disclosure;

FIG. 20 illustrates a diagrammatic view of a CHM file encoding processwherein files of multiple file types are encoded in accordance withvarious embodiments of the present disclosure;

FIG. 21 illustrates a flowchart of a CHM file encoding process whereinfiles of multiple file types are encoded in accordance with variousembodiments of the present disclosure;

FIG. 22 illustrates a diagrammatic view of a CHM file decoding processwherein files of multiple file types are decoded, in accordance withvarious embodiments of the present disclosure;

FIG. 23 illustrates a flowchart of a CHM file decoding process whereinfiles of multiple file types are decoded, in accordance with variousembodiments of the present disclosure;

FIG. 24 illustrates a diagrammatic view of a multiple file type CHMpresentation in accordance with various embodiments of the presentdisclosure;

FIG. 25 illustrates a diagrammatic view of a voice authentication systemutilizing CHM files, in accordance with various embodiments of thepresent disclosure;

FIG. 26 illustrates a flowchart of a voice authentication processutilizing CHM files, in accordance with various embodiments of thepresent disclosure;

FIG. 27 illustrates an account creation abuse prevention process, inaccordance with various embodiments of the present disclosure;

FIG. 28 illustrates a voice-to-text and indexing process utilizing CHMfiles, in accordance with various embodiments of the present disclosure;

FIG. 29 illustrates a diagrammatic view of a database keyword indexsystem, in accordance with various embodiments of the presentdisclosure;

FIG. 30 illustrates a keyword search process, in accordance with variousembodiments of the present disclosure;

FIG. 31 illustrates a diagrammatic view of a combined file encodingprocess with remainders in accordance with various embodiments of thepresent disclosure;

FIG. 32 illustrates a diagrammatic view of a combined file decodingprocess in accordance with various embodiments of the presentdisclosure;

FIG. 33 illustrates a browser window showing a webpage that includes acall to a decoder script according to various embodiments of the presentdisclosure;

FIG. 34 illustrates a flowchart of a CHIF file decoding and blockingprocess in accordance with various embodiments of the presentdisclosure;

FIG. 35 illustrates a CHIF encoding and decoding system in accordancewith various embodiments of the present disclosure;

FIG. 36 illustrates a CHIF encoding and decoding process in accordancewith various embodiments of the present disclosure;

FIG. 37 illustrates a combined file storage and retrieval process inaccordance with various embodiments of the present disclosure;

FIG. 38 illustrates a combined file system compatibility determinationprocess in accordance with various embodiments of the presentdisclosure;

FIG. 39 illustrates a combined file signing and locking process inaccordance with various embodiments of the present disclosure;

FIG. 40 illustrates a combined file tracking process in accordance withvarious embodiments of the present disclosure;

FIG. 41 illustrates a combined file indexing process in accordance withvarious embodiments of the present disclosure;

FIG. 42 illustrates a CHIF file decryption and decoding system inaccordance with various embodiments of the present disclosure;

FIG. 43 illustrates a CHIF file decryption and decoding process inaccordance with various embodiments of the present disclosure;

FIG. 44 illustrates a CHIF file decryption and decoding system inaccordance with various embodiments of the present disclosure;

FIG. 45 illustrates a CHIF file decryption and decoding process inaccordance with various embodiments of the present disclosure;

FIG. 46 illustrates a CHIF file continuous enrichment system inaccordance with various embodiments of the present disclosure;

FIG. 47 illustrates a CHIF file continuous enrichment process inaccordance with various embodiments of the present disclosure;

FIG. 48 illustrates a CHIF previewing architecture in accordance withvarious embodiments of the present disclosure;

FIG. 49 illustrates an example browser window in accordance with variousembodiments of the present disclosure;

FIG. 50 illustrates a CHIF file scarcity creation system in accordancewith various embodiments of the present disclosure;

FIG. 51 illustrates a CHIF file scarcity creation process in accordancewith various embodiments of the present disclosure;

FIG. 52 illustrates an example data mobility and edge computing systemin accordance with various embodiments of the present disclosure;

FIG. 53 illustrates an example data mobility and edge computing processin accordance with various embodiments of the present disclosure;

FIG. 54 illustrates an example travel arrangement offers and edgecomputing system in accordance with various embodiments of the presentdisclosure;

FIG. 55 illustrates an example travel arrangement offers and edgecomputing process in accordance with various embodiments of the presentdisclosure;

FIG. 56 illustrates an example CHIF file with selectively encryptedcontent in accordance with various embodiments of the presentdisclosure;

FIG. 57 illustrates an example CHIF selective encryption system inaccordance with various embodiments of the present disclosure;

FIG. 58 illustrates an example selective encryption process inaccordance with various embodiments of the present disclosure;

FIG. 59 illustrates an example selective decryption process inaccordance with various embodiments of the present disclosure;

FIG. 60 illustrates an example binder creation system in accordance withvarious embodiments of the present disclosure;

FIG. 61 illustrates an example binder creation process in accordancewith various embodiments of the present disclosure;

FIG. 62 illustrates an example secured CHIF container in accordance withvarious embodiments of the present disclosure;

FIG. 63 illustrates an example secured CHIF system in accordance withvarious embodiments of the present disclosure;

FIG. 64 illustrates an example secured CHIF container process inaccordance with various embodiments of the present disclosure; and

FIG. 65 illustrates a diagrammatic view of one embodiment of a systemdevice that may be used within the environment described herein.

DETAILED DESCRIPTION

Referring now to the drawings, wherein like reference numbers are usedherein to designate like elements throughout, various views andembodiments are illustrated and described. The figures are notnecessarily drawn to scale, and in some instances the drawings have beenexaggerated and/or simplified in places for illustrative purposes only.One of ordinary skill in the art will appreciate the many possibleapplications and variations based on the following examples of possibleembodiments.

Digital information such as images, audio, video, text, annotations,etc., is presented and stored as data binaries or bytes. When those databytes are stored in the media depository, they are called files. Whenthey are loaded in the memory of computing devices or are transmitted inthe wire of the network, they are called streams. Blending (encoding)and separating (decoding) operations process the data bytes in thestreams or the files.

Different types of information (images, audio, video, text, documents,programs, etc.) have different data byte structures, called dataformats, when either in the stream or in the file. For example, when animage is stored in the disk or exchanged via the network, if the imageis using JPEG data format, it is the JPEG format or structure of databytes are stored in a file, or transmitted in a stream over the network.Similarly, when audio is stored on the disk or exchanged via thenetwork, if the audio is using an MP3 data format, it is the MP3 formator structure of data bytes that are stored in a file, or transmitted ina stream over a network. So, saving or transmitting an image and animage-related non-image (like audio) has to do two individual processesor tasks, one for image, another for non-image.

The present disclosure provides a unique data stream or file format andstructure—CHM format, having all the data bytes of both the image andthe non-image stream or file, and thus is a combined file or datastream. Along with the CHM format, this disclosure provides theprotocol—CHM formatting protocol, having the method, algorithm, andspecification to blend the image and non-image data streams or filesinto one CHM data stream or file, or separate the CHM data stream orfile back to the image and non-image data streams or files in theiroriginal state.

Referring now to FIG. 1 , there is illustrated a flowchart of oneembodiment of a CHM encoding and decoding process 100 for image andnon-image data streams in accordance with various embodiments of thepresent disclosure. One example includes an image and its description.An encoding method 102 disclosed herein may access image data from anetwork 104 image data generated by a camera 106, or image data fromstorage 108. The encoding method 102 may also access audio data from anetwork 110, audio data from a mic 112, or audio data from storage 114.The encoding method 102 then reads the data bytes from the image andnon-image (audio) stream or file, and blends and writes the data bytesinto one single stream of CHM data 116 or a CHM file with the CHM dataformat. The CHM data 116, which contains both image and non-image data,can be saved into data storage 118 or transmitted to others over anetwork 120 with one single step. And a decoding method 122 disclosedherein separates the image and the audio back to their original statesbefore the image is displayed and the image audio is played.

The image data stream or file format contemplated herein may be anydigital image format. Examples of image data streams or filescontemplated herein include, but are not limited to, JPEG, GIF, TIFF,PNG, Bitmap, RAW, PNM, WEBP, DCM, AVI, MVL, and the like.

The non-image data stream or file format contemplated herein may be anydigital non-image format. Examples of non-image data streams or formatsmay include text data, word processing data, audio data such as MP3,MP4, AIFF, WAY, etc., video data, DCM, AVI, MVL, and the like.

The blending (encoding) and separating (decoding) methods or processesare executed by an application program running in a computing device.The computing devices contemplated herein may include, but are notlimited to, desktop computers, laptop computers, tablet computers,handheld computers, smart phones and other cellular phones, and similarinternet enabled mobile devices, digital cameras, any digital imagegenerating and processing devices, a customized computing deviceconfigured to specifically carry out the methods contemplated in thisdisclosure, and the like. The application program running in thecomputing device contemplated herein may include, but is not limited to,the software executables, the component or library via API called byother software, or the Web APIs or Web Services, and the like.

After they are separated from the CHM format stream or file, the imageor non-image data bytes and their structures or formats are back intheir original states without any changes, so that they can be presentedby their players or processors as the original data streams or fileswithout any changes in quality or functions.

Referring now to FIG. 2 , there is illustrated a data structure orformat 200, referred to as CHM format, for the blended stream or file,in accordance with various embodiments of the present disclosure. Thedata structure or format 200 has two parts: a metadata header section202 that includes metadata bytes at the beginning and a body section 204that includes raw data bytes. The metadata header section 202 recordsthe blending protocol, and other information, such as data size and dataformat, of the original image and non-image data stream or file, and thebase block size of blending and separating. The blended data stream orfile is called “CHM” format stream or “CHM” format file, respectively.

A data process protocol, referred to as CHM formatting, for blending andseparating two data streams or files is provided. The protocol defineshow to determine the block sizes for breaking the image and non-imagedata byte streams or files based on their original data sizes, anddefines the algorithm, steps and sequences to read and write the blocksof image data bytes and non-image data bytes, so as to blend andseparate the image data stream or file and non-image data stream orfile.

Referring now to FIGS. 3A and 3B, there is illustrated a method ofblending or encoding an image stream or file with a non-image stream orfile into a CHM format data stream or file is and decoding of the sameis provided, in accordance with various embodiments of the presentdisclosure. FIG. 3A illustrates a flowchart of one embodiment of anencoding process 300 and FIG. 3B illustrates a flowchart of oneembodiment of a decoding process 302. The process 300 may begin withaccessing both a target image data stream 304 (or target image file) anda target non-image data stream 306 (or target non-image file) by anapplication program configured to perform a CHM encoding operation 308.Once accessed, the program may read data information from both the imagedata stream 304 and the non-image data stream 306, create metadataheader bytes based on the CHM data format, and write the header bytes ata beginning section of a CHM data stream 310 (or CHM file). Next, basedon the data of the image data stream 304 and the non-image data stream306, using the CHM formatting protocol, the program may calculate ablock size for breaking up the image data stream 304 and the block sizefor breaking up the non-image data stream 306. Then, the program mayread one block of data bytes from the image data stream 304, and oneblock of data bytes from the non-image data stream 306, and write thetwo blocks of data bytes in order into the body section of the CHM datastream 310. The program may continue to read the next two blocks of databytes from the image data stream 304 and the non-image data stream 306,and may generate the single CHM data stream 310, and thus may completethe blending or CHM encoding process. The CHM format data stream 310 mayinclude all the data bytes of the image data stream 304 and all the databytes of the non-image data stream 306.

As shown in FIG. 3B, a method of separating or decoding the single CHMdata stream 310 and returning the image data stream 304 and thenon-image data stream 306 is provided. The method, by a programconfigured to perform a CHM decoding operation 314, may begin withaccessing the CHM data stream 310 which may be generated by the blendingmethod described with respect to FIG. 3A. Once accessed, the program mayread the metadata header section of the CHM data stream 310 and thenretrieve the metadata which has the blending protocol and theinformation of the original image data stream 304 and non-image datastream 306. Next, based on the image data stream 304 and the non-imagedata stream 306, with the CHM formatting protocol, the program maycalculate out the block size used to break out the image data stream 304and the block size used to break out the non-image data stream 306.

The program may read one block of bytes from the body section of the CHMdata stream 310 and write the byte block into the image data stream 304,and read another block of bytes from the body section of the CHM datastream 310 and write the byte block into the non-image data stream 306.The program may continue to read the next two blocks of bytes from thebody section of the CHM data stream 310, and write each data byte blockinto the image and non-image data streams 304 and 306 accordingly, andthe program may repeat reading/writing steps until it reaches the end ofthe CHM data stream 310, returning the image and non-image data streams304 and 306 back to their original states without changing any bytes oftheir data and the formats, thus with no change of the qualities orfeatures of them.

Referring now to FIG. 4 , there is illustrated a diagrammatic view ofone embodiment of a CHM file encoding process 400 in accordance withvarious embodiments of the present disclosure. The body section of a CHMfile may be created by blending the bytes of one file or data streamwith one or more other files or data streams. As illustrated in FIG. 4 ,CHM file 402 is created by blending the bytes of an image file 404 and anon-image file 406. It should be understood that the image file 404 andnon-image file 406 may be any file type, even two image files or twonon-image files, and that an image file and a non-image file are usedfor example purposes. In some embodiments, a priority for the blendingalgorithm or protocol is to ensure the bytes from each of the files aredistributed within the CHM file 402 as evenly as possible, even thoughthe image file 404 and the non-image file 406 may not be identical insize.

The example shown in FIG. 4 demonstrates how an algorithm may performthe encoding process in the event that two files happen to have the samenumber of bytes. The bytes of the image file 404 and the non-image file406 are represented in FIG. 4 in hexadecimal format. At a first step 1,the encoder copies a first byte “02” from the image file 404 and writesthe first byte “02” as the first byte of the body section of the CHMfile 402. At second step 2, the encoder copies a first byte “52” fromthe non-image file 406 and writes the first byte “52” as the next byteof the body section of the CHM file 402. At a third step 3, the encodercopies a second byte “16” from the image file 404 and writes the secondbyte “16” as the next byte in the body section of the CHM file 402. At afourth step 4, the encoder copies a second byte “49” from the non-imagefile 406 and writes the second byte “49” as the next (and fourth) bytein the body section of the CHM file 402.

This process of alternating between the image file 404 and the non-imagefile 406 to copy a byte at a time from each continues for all the bytesin the image file 404 and the non-image file 406 until all bytes fromthe image file 404 and the non-image file 406 are written to the CHMfile 402. At a penultimate step n−1, the encoder copies a last byte “00”from the image file 404 and writes the last bye “00” to the body sectionof the CHM file 402. At a last step n, the encoder copies a last byte“22” from the non-image file 406 and writes the last byte “22” to thebody section of the CHM file 402. After the last step n, the CHM file402 is completed, the CHM file 402 now containing all bytes from theimage file 404 and the non-image file 406 blending together. The CHMfile 402 thus may be the same file size as the sum of the file sizes ofthe image file 404 and the non-image file 406, as the CHM file containsthe bytes of each of the image file 404 and the non-image file 406, withlittle other information added.

Referring now to FIG. 5 , there is illustrated a flowchart of a CHMcreation process 500 in accordance with various embodiments of thepresent disclosure. The process 500 begins at step 502 where an encoderor application analyzes properties of each of an image file and anon-image file. At step 504, the encoder determines an appropriate byteblock size for each of the image file and the non-image file wherein thebyte block sizes are based on a file size ratio that is the size of theimage file to the size of the non-image file. For example, as in theexample illustrated in FIG. 4 , the image file 404 and the non-imagefile 406 both contain twenty-five bytes of data. Therefore, there is a1:1 ratio between the image file 404 and the non-image file 406.

In the case of a 1:1 ratio, the byte block size for both the image fileand the non-image file may be one byte, in order to even distribute eachbyte from the image file and the non-image file within the CHM file. If,for example, a 3:1 ratio existed between the number of bytes of theimage file to the non-image file, three bytes would be copied from theimage file and written to the CHM file for every one byte from thenon-image file, or vice versa in the case of a 1:3 ratio. In the eventthat the number of bytes of the image file and the non-image file cannotbe expressed easily as a ratio, other methods for determining the byteblock size may be performed, as described herein.

After the byte block size for the image file and for the non-image fileis determined, the process flows to step 506. At step 506, the encodercreates a metadata header for a new CHM file based on the image file andnon-image file properties. The metadata header may also includeinformation concerning the byte block size of each of the image file andthe non-image file, so that a decoder may use the metadata headerinformation at a later time to determine how the CHM file should bedecoded. At step 508, the encoder reads a byte block from the image fileand writes the byte block to a body section of the new CHM file. At step510, the encoder reads a byte block from the non-image file and writesthe byte block to the CHM file. The process 500 flows to decision block512, where it is determined whether the last image file byte block hasbeen written to the CHM file. If not, the process flows back to step 508to write another byte block from the image file, to write anothernon-image file byte block at step 510, and return to decision block 512to again determine whether the last image file byte block has beenwritten to the CHM file. If at decision block 512 it is determined thatthe last byte block has been written to the CHM file, the process 500flows to step 514 to read the last byte block from the non-image fileand write the byte block to the CHM file. The process 500 ends with step516, where the encoded CHM file is stored.

Referring now to FIG. 6 , there is illustrated a diagrammatic view ofone embodiment of a CHM file decoding process 600 in accordance withvarious embodiments of the present disclosure. A CHM file 602 mayinclude a plurality of bytes blended into the CHM file previously duringa CHM encoding operation. The plurality of bytes in the CHM file 602 isshown in FIG. 6 as what may be the final result of the encoding processillustrated in FIG. 4 , with the plurality of bytes in the CHM file 602including all the bytes from the image file 404 and the non-image file406. To decode the CHM file and recreate an image file 604 and anon-image file 606, a decoder may determine the byte block size used tocreate the CHM file 602 and begin reading and writing byte blocks fromthe CHM file 602 to the image file 604 and the non-image file 606.

At a first step 1, the decoder reads a first byte “02” from the CHM file602 and writes the first byte “02” to the image file 604 as the firstbyte of the image file 604. At a second step 2, the decoder reads asecond byte “52” from the CHM file 602 and writes the second byte “52”to the non-image file 606 as the first byte of the non-image file 606.At a third step 3, the decoder reads a third byte “16” from the CHM file602 and writes the third byte “16” to the image file as the second byteof the image file 604. At a fourth step 4, the decoder reads a fourthbyte “49” from the CHM file 602 and writes the fourth byte “49” as thesecond byte of the non-image file 606. This pattern continues until allbytes from the CHM file are read and written to the image file 604 andthe non-image file 606. At a penultimate step n−1, the decoder writes apenultimate byte “00” to the image file 604 as the last byte of theimage file 604. At a last step n, the decoder writes a last byte “22” tothe non-image file as the last byte of the non-image file 606. Afterstep n, the image file 604 and the non-image file 606 are completed. Theimage file 604 and the non-image file 606 may be exact copies of theimage file and non-image file that were used during creation andencoding of the CHM file 602, such as the image file 404 and non-imagefile 406.

Referring now to FIG. 7 , there is illustrated a CHM decoding process700 in accordance with various embodiments of the present disclosure.The process 700 begins at step 702 where a decoder reads a metadataheader of a CHM file. The metadata header may contain informationrelated to the original data streams used to create the CHM file, suchas byte block sizes for the files. At step 704, the decoder determines abyte block size for each of the image file byte blocks the non-imagefile byte blocks included within the CHM file. At step 706, the decoderreads a byte block from the CHM file and writes the byte block to animage file. At step 708, the decoder reads a byte block from the CHMfile and writes the byte block to a non-image file. At decision block710, the decoder determines whether the last byte block has been writtento the image file. If not, the process flows back to step 706 and thedecoder reads the next image file byte block from the CHM file andwrites the byte block to the image file, and then moves to step 708again to read the next non-image byte block from the CHM file and writethe byte block to the non-image file.

If at decision block 710, it is determined that the last image file byteblock has not been written to the image file, the process flows to step712 where the decoder reads a last byte block from the CHM file andwrites the byte block to the non-image file. After step 712, the imagefile and the non-image file are completed. The image file and thenon-image file may be exact copies of the image file and non-image filethat were used during creation and encoding of the CHM file. The process700 ends with step 714 where the decoded image file and the decodednon-image file are stored.

Referring now to FIG. 8 , there is illustrated a diagrammatic view of anembodiment of a CHM file encoding process 800 where the files to beblended into a CHM file 802 do not have an equal number of bytes, inaccordance with various embodiments of the present disclosure. FIG. 8illustrates an image file 804 having a total of 25 bytes and a non-imagefile having a total of 72 bytes. The protocol for CHM encoding anddecoding may call for the bytes of the files combined into the CHM file802 to be as evenly distributed as possible. For example, the protocolmay be written to avoid having multiple byte blocks of a single file bebunched together. For instance, if only one byte was written from theimage file 804 and the non-image file 806 at a time, such as thatillustrated in FIG. 4 , the resulting CHM file would have the first 50bytes be evenly blended from the image file 804 and the non-image file806, but the last 47 bytes of the CHM file would all be non-image filebytes. While the protocol may allow for such an encoding algorithm, theprotocol may dictate a more even distribution of bytes.

As shown in FIG. 8 , the encoder determines that the byte block size forthe image file 804 is one byte, while the byte block size for thenon-image file 806 is three bytes. This may be determined by amathematical operation such as

${b = \left\lceil \frac{y}{x} \right\rceil},$

or b=ceil(y/x), where y is the number of bytes for the file having alarger number of bytes, x is the number of bytes for the file having theleast number of bytes, and b is the block size for the file having thelarger number of bytes. So, for example, since non-image file 806 has 72bytes, and image file 804 has 25 bytes, b=3. If more than two files areto be written into the CHM file, this operation could be performed forevery file that has more bytes than the file with the fewest bytes. Forexample, if another non-image file was to be blended into the CHM file802, the block size for the non-image file 806 would still be 3 and theblock size for the image file 804 would still be 1. If the othernon-image file has 38 bytes, for example, b=2 for the other non-imagefile. The encoder would then alternate between the three files, writinga byte from the image file 804 to the CHM file, three bytes from thenon-image file 806 to the CHM file, and two bytes from the othernon-image file to the CHM file, until all bytes are copied to the CHMfile.

As shown in FIG. 8 , the encoder, at a first step 1, reads a first byte“02” from image file 804 and writes it as the first byte of the CHM file802. At a second step 2, the encoder reads the first three bytes “52 4946” from the non-image file 806 and writes the bytes to the CHM file802. At a third step 3, the encoder reads a second byte “16” from theimage file and writes the second byte to the CHM file 802 after thealready written bytes. At a fourth step 4, the encoder reads the nextthree bytes in the non-image file 806 and writes the three bytes to theCHM file 802. This process of switching between the image file 804 andthe non-image file 806 continues until all bytes from the image file 804and the non-image file 806 are written to the CHM file 802. At apenultimate step n−1, the last three bytes of the non-image file 806 arewritten to the CHM file 802. At a last step n, the last byte of theimage file 804 is written to the CHM file 802.

It should be noted that the number of bytes used as an example in FIG. 8cause a byte block from the image file 804 to be the first and lastwrite to the CHM file 802. It will be understood that depending on thenumber of bytes in the files, the algorithm may not perfectly evenlydistribute byte blocks. In the example of FIG. 8 , there were 25 byteblock writes from the image file 804 and 24 byte block writes from thenon-image file 806. However, the algorithm still distributed them in aneven manner. In some embodiments, an assigned byte block size causethere to be a remainder of bytes left over. For example, if there were125 bytes in one file and 15 bytes in another, b=9. So, there would be15 writes from the file with 15 bytes (one byte per byte block).However, after the 13^(th) write for the file with 125 bytes (at 9 bytesper byte block), 117 bytes from the file would have been written intothe CHM file after that 13^(th) write. Since there are 125 bytes in thefile, there are 8 bytes remaining to be written after the 13^(th) write.In this scenario, for the 14^(th) write from the file having 125 bytes,the encoder may determine that a full byte block does not remain in thefile, and simply write the last 8 bytes into the CHM file as the lastbyte block written from the file with 125 bytes. In other embodiments,the encoder may divide the last 8 bytes into two byte blocks of 4 byteseach, so that there is a 14^(th) and a 15^(th) read/write operation forboth the file with 15 bytes and the file with 125 bytes.

In some embodiments, to limit the time spent performing read/writeoperations, a multiplier may be applied. For example, if the number ofbytes for the files are 25 and 72, as in FIG. 8 , b=3, and the encodermay write one byte at a time for the image file 804 and three bytes at atime for the non-image file 806. However, this causes there to be atotal of 49 read/write operations. A multiplier may be applied todecrease the number of read/write operations, which may be useful inspeeding up the process and limiting strain on resources, especially forfiles having a large number of bytes. For example, a 2× multiplier maybe applied, increasing the byte block size for the image file 804 to 2,and the byte block size for the non-image file 806 to 6. Thus, theencoding process would only require twenty-five read/write operations(including any extra writes for remainders).

Referring now to FIGS. 9A and 9B, there is illustrated a flowchart of aCHM encoder process 900 in accordance with various embodiments of thepresent disclosure. The process begins at step 902 where an encoderanalyzes the properties of each of an image file and a non-image file.It will be understood that the files may be any file type orcombinations of file types, and that use of an image file and non-imagefile is just for example purposes. At step 904, the encoder creates ametadata header using information found during step 902 concerning theimage file and the non-image file. At step 906, the encoder divides thenumber of bytes of the file that has a larger number of bytes by thenumber of bytes of the file that has a smaller number of bytes. Atdecision block 908, it is determined whether the division performed instep 906 results in a number having a remainder. If so, the process 900flows to step 910 where the result of the division in step 906 isrounded up to the next integer (2.5 to 3, for example) and a byte blocksize that is equal to that integer is assigned for the file having alarger number of bytes. Steps 906-910 may be practically in the encoderprogram by performing a ceil( ) function or by performing integerdivision and adding one to the result. At step 912, the byte block sizefor the smaller file is set to one. The process then flows to step 914.

If at decision block 908 it is determined there is no remainder, theprocess flows to step 916 where the byte block sizes for the image fileis set based on a ratio of the number of bytes of the image file to thenumber of bytes of the non-image file. For example, if the image filehas 18 bytes and the non-image file has 27 bytes, the ratio is 2:3, sothe encoder would assign a byte block size of 2 to the image file and abyte block size of 3 to the non-image file. The process then flows tostep 914. At step 914, a speed multiplier is set such as describedherein, to optionally speed up the encoding process and reduce thenumber of read/write operations. If not needed, the speed multiplier canbe set to 1 to keep the assigned byte block sizes.

The process 900 then flows to decision block 918, where it is determinedwhether the last image file byte block has been written to the CHM file.If not, the process 900 flows to step 920. At step 920, the encoderreads a byte block from the image file and writes the byte block to theCHM file. At decision block 922, it is determined whether the lastnon-image file byte has been written to the CHM file. If not, theprocess 900 flows to step 924. At step 924, the encoder reads a byteblock from the non-image file and writes the byte block to the CHM file.At decision block 926, it is determined whether all bytes from both theimage and the non-image file have been written to the CHM file. If not,the process 900 flows back to decision block 918. If at decision block918 it is determined that the last image file byte has been written tothe CHM file, the process flows to decision block 922. If at decisionblock 922 it is determined that the last non-image file byte has beenwritten to the CHM filed, the process flows to decision block 926. If atdecision block 926 it is determined that all bytes have been written,the process flows to step 928. At step 928, the encoded CHM filed isstored.

Referring now to FIG. 10 , there is illustrated a diagrammatic view ofan embodiment of a CHM file decoding process 1000 where the files to bedecoded from a CHM file 1002 do not have an equal number of bytes, inaccordance with various embodiments of the present disclosure. Theheader data of the CHM file may indicate how the CHM file 1002 wascreated, and the byte block sizes used for the files making up the CHMfile 1002. The example shown in FIG. 10 is a decoding process of the CHMfile illustrated in FIG. 8 . At a first step 1, a first byte “02” isread from the CHM file 1002 and written to an image file 1004. At asecond step 2, three bytes “52 49 46” are read from the CHM file 1002and written to a non-image file 1006. At a third step 3, a byte “16” isread from the CHM file 1002 and written to the image file 1004. At afourth step 4, three bytes “46 24 08” are read from the CHM file 1002and written to the non-image file 1006. The process 1000 continuesalternating writing one byte to the image file 1004 and writing threebytes to the non-image file 1006 in order to write all bytes from theCHM file 1002. At a penultimate step n−1, the first three bytes “ce 1a0d” of the last four bytes of the CHM file are written to the non-imagefile 1006. At a last step n, the last byte “00” of the CHM file iswritten to the image file 1004. After step n, the image file 1004 andthe non-image file 1006 are completed. The image file 1004 and thenon-image file 1006 may be exact copies of the image file and non-imagefile that were used during creation and encoding of the CHM file 802,such as the image file 804 and non-image file 806.

Referring now to FIG. 11 , there is illustrated a flowchart of a CHMdecoding process 1100 in accordance with various embodiments of thepresent disclosure. The process 1100 begins at step 1102 where a decoderreads the metadata header of a CHM file. At step 1104, the decoderdetermines a byte block size for each of the image file blocks and thenon-image file blocks included within the CHM file.

The process 1100 then flows to decision block 1106, where it isdetermined whether the last image file byte block has been written to animage file. If not, the process 1100 flows to step 1108. At step 1108,the decoder reads a byte block from the CHM file and writes the byteblock to the image file. At decision block 1110, it is determinedwhether the last non-image file byte has been written to a non-imagefile. If not, the process 1100 flows to step 1112. At step 1112, thedecoder reads a byte block from the CHM file and writes the byte blockto the non-image file. At decision block 1114, it is determined whetherall bytes from the CHM file have been written to the image file andnon-image file. If not, the process 1100 flows back to decision block1106. If at decision block 1106 it is determined that the last imagefile byte has been written to the image file, the process flows todecision block 1110. If at decision block 1110 it is determined that thelast non-image file byte has been written to the non-image file, theprocess flows to decision block 1114. If at decision block 1114 it isdetermined that all bytes have been written to the CHM file, the processflows to step 1116. At step 1116, the decoded image and non-image filesare stored.

Referring now to FIG. 12 , there is illustrated a diagrammatic view of aserver-side CHM file decoding and transmission system 1200 in accordancewith various embodiments of the present disclosure. The system 1200includes a server 1202 that includes a CHM codec 1204. The CHM codec1204 may be a program for encoding and/or decoding CMH files or datastreams. The server 1202 may receive a request from a mobile device1206, the request being sent over a network 1208. The server may have aCHM file stored thereon. The server 1202 may be a web server configuredto present the CHM file as part of a webpage. For instance, the webpagemay be an online store page for a product. The CHM file may be used tostore an images or images of the product and accompanying audio todescribe the product, as well as other file types and information. Whenthe webpage loads on the mobile device 1206, a user of the mobile device1206 may view images of the product and hear audio describing theproduct. It will be understood that this is but one example, and thesystems and processes described herein may be used to present images andaccompanying audio in other scenarios as well.

In some embodiments, the server 1202 may decode the CHM file beforesending the separate files or data streams over the network 1208 to themobile device 1206. This allows for the webpage and the contents of theCHM file to be viewed or accessed without the mobile device 1206requiring a CHM codec or browser plugin to decode the CHM file. In otherembodiments, the mobile device 1206 may include such a codec or plugin,in which case the server may transmit the CHM file to the mobile device1206, and the mobile device 1206 would perform a decoding process on theCHM file. As shown in FIG. 12 , the server 1202 provides a CHM datastream 1210 to the CHM codec 1204, the CHM data stream 1210 being astream from the CHM file stored on the server. The CHM codec 1204decodes the CHM data stream 1210 into separate files or data streamsthat were originally used during creation of the CHM file. As oneexample, and as illustrated in FIG. 12 , the CHM codec 1204 may decodethe CHM data stream 1210 into a separate image data stream 1212 and aseparate non-image data stream 1214. The server 1202 may then transmitthe image data stream 1212 and the non-image data stream 1214 over thenetwork 1208 to the mobile device 1206, for use by the mobile device.

Referring now to FIG. 13 , there is illustrated a browser window 1300showing a webpage 1302 including an image 1304 with accompanying audio1306 presented on the webpage 1302, in accordance with variousembodiments of the present disclosure. As described with respect to FIG.12 , a CHM file may be received by a device on which a browser or othermeans of accessing web content resides. A server may decode a stored CHMfile and then transmit the separate streams to the device, the separatestreams providing content to be presented on the webpage 1302. Once thewebpage 1302 is loaded in the browser window 1300 the image 1304 andother webpage content such as user interaction buttons 1308 and textualproduct information 1310 may be presented. The audio 1306 may beginplaying once the website is loaded, or the audio 1306 may only play upona trigger instructing the audio 1306 to play. For example, the audio1306 may only play when a mouse cursor 1312 is placed over the image1304, when the image 1304 is clicked on, or when the image 1304 isscrolled such that the image 1304 is viewable in the browser window1300, with the audio 1306 stopping once the user scrolls away from theimage 1304.

Presenting audio with an image in this way offers a more efficient meansof providing audio information with an image. Typically, if one wishesto provide image content in association with audio content, one wouldcreate a video file, such as an MP4 file, and lay an audio track overthe image. This may be an inefficient method of associating audiocontent with an image because if the goal is to provide audio contentwith one or more still images, rather than with moving video content,creating a video file to achieve such creates a bigger file than needed,as video files are commonly much larger than an image or an audio file,even when the image or audio file sizes are combined. The CHM file isthe same or a similar size to the combined size of the image and audiofiles, and thus provides a more efficient file type that takes up lessstorage, is transmitted faster, etc. It will be understood that this maybe the case for other file types as well, such as if a text document wasalso included in the CHM file, the size of the CHM file would onlyincrease in an amount close to that of the size of the text document.

Referring now to FIG. 14 , there is illustrated a flowchart of aserver-side CHM decoding process 1400 in accordance with variousembodiments of the present disclosure. The process 1400 begins at step1402 when the server receives a request for a webpage. At decision block1404, it is determined whether a CHM file is associated with therequested webpage, such as if content to be loaded on the webpage isencoded within the CHM file. If not, the process 1400 ends at step 1406,where the server transmits the requested webpage content. If at decisionblock 1404 it is determined that there is a CHM file associated with therequested webpage, the process 1400 moves to step 1408. At step 1408,the server decodes the CHM file, or CHM files, stored on the server thatis associated with the requested webpage. The decoding process dividesthe CHM file into separate data streams, such as an image data streamand an audio data stream.

The process 1400 then flows to step 1410, where the server transmits therequested webpage content, including the data streams separated from theCHM file, such as the image data stream and audio data stream. At step1412, the webpage, including the separated data stream content, such asthe image, is loaded on a mobile device. At step 1414, the image, nowloaded as part of the webpage in a browser or other means of displayingweb content, is activated. Such activation may be a tap on atouchscreen, a click, a mouse rollover, a scrolling operation thatbrings the image into view, or other means. At step 1416, audio playbackbegins from the audio data stream.

Referring now to FIG. 15 , there is illustrated a medical imaging anddictation CHM file creation system 1500 in accordance with variousembodiments of the present disclosure. A medical imaging process (suchas an MRI, X-ray, etc.) may create a high resolution medical imagingfile 1502. A medical professional 1504, such as a doctor, surgeon,medical assistant, etc., may review the medical imaging file 1502 inorder to provide an opinion on what is shown in the medical imaging file1502. During such a review, the medical professional 1504 may create anotes file 1506. The notes file 1506 may be a text document includingwritten or textual notes, the notes file 1506 may be an audio dictationwhere the medical professional's speech is recorded, or other means ofstoring the medical professional's notes. Once the notes file iscreated, the medical imaging file 1502 and the notes file 1506 areprovided to a CHM encoder 1508. The CHM encoder 1508 may be configuredto perform a CHM encoding operation only, or may include a codec forboth encoding and decoding of CHM files. The CHM encoder 1508 creates aCHM file 1510, including both the medical imaging file 1502 bytes andthe notes file 1506 bytes. The CHM file 1510 may be stored on a server1512 or another device for storage, until needed at a later time.

For example, a medical facility, such as a medical specialist seeing thepatient after the high resolution medical image was created, may requestto see the high resolution medical imaging file 1502, along with thenotes file 1506. Upon such a request, the server 1512 may transmit theCHM file 1510 over a network 1514 to a medical facility device 1516belonging to the requesting medical facility. The medical facilitydevice 1516 may include or be operatively connected to a CHM decoder1518. The CHM decoder 1518 may be configured to perform a CHM decodingoperation only, or may include a codec for both encoding and decoding ofCHM files. Upon receiving the CHM file 1510 from the server 1512 by themedical facility device 1516, the CHM decoder 1518 may decode the CHMfile 1510 to separate from the CHM file 1510 the high resolution medicalimaging file 1502 and the medical professional notes file 1506. The CHMfile 1510 may only be a size at or similar to the combined sizes of thehigh resolution medical imaging file 1502 and the notes file 1506. Insome embodiments, no compression may be applied during creation of theCHM file 1510, so as to avoid in loss of image quality of the medicalimaging file 1502 from a compression process. The CHM file 1510 allowsfor the imaging file 1502 to be transmitted and separated back out ofthe CHM file 1510 in its original, high resolution, state, so thatanother medical professional can review the image without any loss inquality. During review of the medical imaging file 1502, the notes file1506 may be reviewed at the same time, such as listening to a dictationperformed by the medical professional 1504 while viewing the imagingfile 1502.

Referring now to FIG. 16 , there is illustrated a medical imaging anddictation CHM file creation process 1600 in accordance with variousembodiments of the present disclosure. The process 1600 begins at step1602, where a medical imaging process is completed, such as an MRI,X-ray, etc. At step 1604, a high resolution medical image file iscreated of a subject of the medical imaging process. At step 1606, amedical professional performs a dictation or other means of creatingnotes providing an analysis or opinion of what the medical professionalmay see in the high resolution medical image file. At step 1608, adictation file is created to store the dictation or other notes createdin step 1606.

At step 1610, a CHM encoder receives the medical image file and thedictation file. At step 1612, the CHM encoder encodes the medical imagefile and the dictation file into a CHM file. At decision block 1616, itis determined whether the medical image is to be reviewed, such as by adoctor or other medical professional. If not, the CHM file may be storeduntil such time as the image is to be reviewed. If so, the process 1600flows to step 1618. At step 1618, a CHM decoder decodes the CHM file inorder to separate the CHM file into the individual files or data streamsused to create the CHM file, in this case the medical image file and thedictation file. At step 1620, the medical image file is viewed whilealso accessing the dictation file, such as listening to audio playbackfrom the dictation file while viewing the medical image. The processthen ends at step 1622.

A binder file may also be provided. A binder file may incorporatemultiple CHM files within the binder file in order to provide filegroupings defined by the CHM file. While a CHM file may include bytesfrom any number of files, as described herein, a binder file can be usedto transmit a series of CHM files where each of the CHM files is createdfrom a number of associated files. For instance, CHM files stored in abinder file may each include an image data stream and an audio datastream. When the binder file is accessed, the first CHM file may bedecoded to present to a user the image from the first CHM file, whilealso playing the audio from the first CHM file. Once audio playback iscomplete, the next CHM file in the binder file may be decoded so thatthe image and audio from the next CHM file can be presented. Thus, thebinder file allows for a series of images, or a presentation, to beprovided to a user. The binder file may include CHM files having anytypes of file data streams stored therein, such as text files, documentfiles, video files, executable files, etc., in order to provide a fullsuite of information and content to a user.

Referring now to FIG. 17 , there is illustrated a diagrammatic view of aCHM binder file creation process 1700 in accordance with variousembodiments of the present disclosure. There is shown a plurality ofimage files associated with an audio file. A first image file 1702 isassociated with a first audio file 1704, a second image file 1706 isassociated with a second audio file 1708, and an n^(th) image file 1710is shown associated with an n^(th) audio file 1712. The plurality ofimage and audio files are processed by a CHM encoder 1714 to create aplurality of CHM files, each of the plurality of CHM files created froman image file and its associated audio file. A first CHM file 1716 iscreated from the first image file 1702 and the first audio file 1704. Asecond CHM file 1718 is created from the second image file 1706 and thesecond audio file 1708. An nth CHM file 1720 is created from the nthimage file 1710 and the nth audio file 1712. Each of the plurality ofCHM files may then be stored within a binder file 1722.

Referring now to FIG. 18 , there is illustrated a diagrammatic view of aCHM binder file decoding process 1800 in accordance with variousembodiments of the present disclosure. A binder file 1802 may include aplurality of CHM files. As illustrated in FIG. 18 , the binder file 1802includes a first CHM file 1804, a second CHM file 1806, and a nth CHMfile 1808, indicating that any number of CHM files may be included inthe binder file 1802. To decode and present the contents of theplurality of CHM files, the CHM files may be decoded by a CHM codec1810, such as the decoding processes described herein. The CHM codec1810 each of the plurality of CHM files into separate files or datastreams that may be identical to the files or data streams used duringthe encoding process to create the CHM file. For example, decoding ofthe first CHM file 1804 provides a first image file 1812 and a firstaudio file 1814.

Once the first CHM file 1804 is decoded, audio from the first audio file1814 may be played while the image of the first image file 1812 ispresented to a user. Upon completion of playback of the first audio file1814, or if the user performs an action that ends the audio playback orotherwise advances the process, the CHM codec 1810 may decode the secondCHM file 1806 to provide a second image file 1816 and a second audiofile 1818. Once the second audio file 1818 has completed playback, theCHM codec 1810 decodes the nth CHM file 1808, producing an nth imagefile 1820 and an nth audio file 1822. In this way, a series of contentmay be presented to a user.

Referring now to FIG. 19 , there is illustrated a binder file creationand decoding process 1900 in accordance with various embodiments of thepresent disclosure. The process 1900 begins at step 1902 where an imagefile and an audio file associate with the image file are created. Atstep 1904, the image file and the associated audio file are encoded intoa CHM file. At step 1906, the CHM file created in step 1904 is savedinto a binder file. At decision block 1908, it is determined whetheradditional image and audio files are to be created. If so, the process1900 moves back to step 1902 to create another image file and associatedaudio file, encode the image file and audio file into a CHM file at step1904, and save the CHM file into a binder file. If at decision block1908 it is determined that no additional image and audio files are to becreated, and thus no more CHM files encoded and stored in the binderfile, the process 1900 flows to step 1910.

At step 1910, a CHM file is removed from the binder file. At step 1912,the CHM file removed from the binder file in step 1910 is decoded intoseparate image and audio streams. At step 1914, audio playback from theaudio data stream is performed while the image from the image datastream is presented. At decision block 1916, it is determined whetheradditional content stored in the binder file is to be accessed. If so,the process 1900 flows back to step 1910 to remove another CHM file fromthe binder file, decode the CHM data stream at step 1912, and playbackaudio at step 1914. If at decision block 1916 it is determined that noadditional content is to be accessed, the process 1900 ends at end block1918.

Referring now to FIG. 20 , there is illustrated a diagrammatic view of aCHM file encoding process 2000 wherein files of multiple file types areencoded in accordance with various embodiments of the presentdisclosure. A CHM file may include the bytes of any number of files ofvarious file types blended into the CHM file and later decoded out ofthe CHM file. FIG. 20 illustrates a plurality of files each having afile type being encoded into a single CHM file. There is shown a file ofa first file type 2002, a file of a second file type 2004, a file of athird file type 2006, and a file of an nth file type 2008, indicatingthat there may be any number of files, each having a different filetype. It will be understood that multiple files of the same file typemay also be included, such as including three image files, two audiofiles, and four text documents, for example. The plurality of files maybe sent to a CHM encoder 2010. The CHM encoder 2010 encodes all of theplurality of files into a CHM file 2012. The encoding process may beperformed in as described herein to evenly distribute the bytes of eachof the files within the CHM file 2012.

Referring now to FIG. 21 , there is illustrated a flowchart of a CHMfile encoding process 2100 wherein files of multiple file types areencoded in accordance with various embodiments of the presentdisclosure. The process 2100 begins at step 2102, where a first file ofa first file type is retrieved. At step 2104, another file of anotherfile type is retrieved. At decision block 2106, it is determined whetheradditional files and files of other types are to be included in theencoding process. If so, the process moves back to step 2104 to retrieveanother file. If at decision block 2106 it is determined that noadditional files or file types are to be included in the encodingprocess, the process flows to step 2108. At step 2108, a byte block ofthe first file of the first file type is encoded. At step 2110, a byteblock of the next file retrieved is encoded. At decision block 2112, itis determined whether all byte of all files have been encoded. If not,the process 2100 moves back to step 2110 to encode the next byte block.In step 2110, the “next” file may be a file that the encoding processreturns to in order to write the next byte block from that file. Forinstance, if the process 2100 is encoding three files into a CHM file,step 2108 may be performed to encode the first byte block of the firstfile, step 2110 may be performed to write the first byte block from thesecond file, and at decision block 2112 it is determined that not allbytes have yet been written. The process 2100 would then move back tostep 2110 to write the first byte block of the third file. Afterdetermining again at decision block 2112 that not all bytes have beenwritten, the process 2100 may move back to step 2110 to write the nextbyte block of the first file, and so on until all bytes from all filesare encoded. Once at decision block 2112 it is determined that all bytesfrom all files have been decoded, at step 2114 the CHM file is stored.The process 2100 then ends at step 2116.

Referring now to FIG. 22 , there is illustrated a diagrammatic view of aCHM file decoding process 2200 wherein files of multiple file types aredecoded, in accordance with various embodiments of the presentdisclosure. A CHM file 2202 containing the blended bytes of a pluralityof files of various file types is processed by a CHM decoder 2204. TheCHM decoder 2204 may perform a decoding operation such as thosedescribed herein. The CHM decoder 2204 decodes the CHM file 2202 into aplurality of data streams of various file types. There is shown in FIG.22 a data stream of a first file type 2206, a data stream of a secondfile type 2208, a data stream of a third file type 2210, and a datastream of an nth file type 2212.

Referring now to FIG. 23 , there is illustrated a flowchart of a CHMfile decoding process 2300 wherein files of multiple file types aredecoded, in accordance with various embodiments of the presentdisclosure. The process 2300 begins at step 2302 where a CHM file isretrieved. At step 2304, a byte block of a first file of a first filetype is decoded from the CHM file. At step 2306, a byte block of a nextfile is decoded. At decision block 2308, it is determined whether allbytes of all the files have been decoded. If not, the process 2300 movesback to step 2306 to decode the next byte block of the next file. If itdecision block 2308 it is determined that all bytes have been decoded,the process 2300 flows to step 2310. At step 2310, the decoded files maybe presented or played back to a user. The process then ends at endblock 2312.

Referring now to FIG. 24 , there is illustrated a diagrammatic view of amultiple file type CHM presentation 2400, in accordance with variousembodiments of the present disclosure. The presentation 2400 may bepresented to a user after or during a successful CHM decoding operationsuch as that described herein. The presentation 2400 shows that a CHMfile may allow for a full suite of files and information to be presentedto a user. FIG. 24 illustrates a device 2402 including a screen 2404.The presentation 2400 presents on the screen 2404 a plurality of filesof various file types For example, and as shown in FIG. 24 , there maybe presented an image 2406, a presentation document 2408 (such as aMicrosoft Powerpoint document), a text document 2410, and a spreadsheetdocument 2412. While a user is viewing these files of differing filetypes, audio 2414 may be played back to provide the user additionalinformation concerning what the user is seeing on the screen 2404.

Referring now to FIG. 25 , there is illustrated a diagrammatic view of avoice authentication system 2500 utilizing CHM files, in accordance withvarious embodiments of the present disclosure. The system 2500 includesan authentication server 2502. The authentication server 2502 mayinclude an audio conversion engine 2504 and a database 2506. Theauthentication server 2502 may use the audio conversion engine 2504 torecognize speech and convert audio to text and may use the database 2506to recognize user speech patterns and commands to perform voiceauthentication for various purposes. The database 2506 may include usercredentials 2508 and training data 2510. The user credentials 2508 mayinclude user authentication data such as usernames, passwords, answersto security questions, and other information that the system might useto authenticate a user. The training data 2510 may include user-specificdata that has been accumulated over time as the user has utilized theauthentication server 2502 to authenticate items. The training data 2510may record the speech patterns of a user to recognize the specificspeech patterns for the user. The authentication server may performvoice recognition through acoustic and/or language modeling to determinewhat a user is saying in audio provided to the authentication server2502.

The authentication server 2502 may receive a CHM file 2512 to be used inauthentication. The authentication may be performed for various reasons,such as to authenticate a login on a website, authenticate access todocuments, etc. For example, a user may be provided a contract to beviewed only by the user. In order to access the contract, the user mayneed to first authenticate the user's identity by providing a voicedcommand or password. As another example, a website or other serviceprovided to the user such as a mobile device app, etc., that allows foraccounts to be created may use voice authentication to login. If a userwho previously created an account with a website is for some reasonbanned from the website, the server may keep a record of the user'svoice authentication data. If that user ever tries to create a newaccount on the website to circumvent the ban, the website may ask forvoice authentication to be set up by the user. The server may then checkbanned user voice authentication or training data in order to determineif the user attempting to create the new account has been bannedpreviously. If so, the account creation may be denied.

A CHM encoder 2514 may receive a textual authentication data stream 2516including items such as a username password, etc., and may also receivea voice data stream 2518. The CHM encoder 2514 may encode the textualauthentication data stream 2516 and the voice data stream 2518 into theCHM file 2512. The CHM file 2512 may be transmitted over a network 2520to the authentication server 2502 by a mobile device (not shown). Uponreceiving the CHM file 2512, the authentication server 2502 may decodethe CHM file 2512 to separate the textual authentication data stream2516 from the voice data stream 2518. The authentication server 2502 maythen compare the textual authentication data stream 2516 to the usercredentials 2508 stored on the database 2506. If the textualauthentication data stream 2516 provided matches the user credentials2508 stored on the database 2506, the system may then perform speechrecognition on the speech data proved by the user.

The speech may be received initially through a microphone and the analogsound waves are converted into a digital format by an analog-to-digital(A/D) converter. The digital data may be converted into a spectrogramshowing how the frequencies of sound change in intensity over time usingfast Fourier transform FFT. The data may then be separated into acousticframes. The speech data may be analyzed for specific phonemes, phones,formants, etc. to recognize what is being said in the speech data. Thespeech patterns may also be analyzed to determine who is speaking in therecording. Over time, a user's training data is updated to moreeffectively recognize speech from that user. The system 2500 may alsoutilize neural networks to assist in speech recognition.

Referring now to FIG. 26 , there is illustrated a flowchart of a voiceauthentication process 2600 utilizing CHM files, in accordance withvarious embodiments of the present disclosure. The process 2600 beginsat step 2602 where a user attempts to access a voice authenticated item.At step 2604, the voice capture is activated and a recording of theuser's voice is captured and stored as a voice audio file. At step 2606,the created voice audio file is encoded with other user authenticationinformation into a CHM file, as described herein. The other userauthentication information may include usernames, passwords, personalinformation, security questions and answers, etc. The other userauthentication information may be in a text data type of format. At step2608, the encoded CHM file is transmitted to an authentication server.

At step 2610, the authentication server decodes the CHM file to separatethe voice audio data stream from the other user authenticationinformation data stream. At step 2612, the authentication servercompares the voice audio data with voice training data on an associateddatabase and may also perform speech recognition processes using anaudio conversion engine. Also, at step 2612, the authentication servermay compare the other user information data with user data stored on theauthentication server or associated database. At decision block 2614, itis determined whether there is a match between the other userauthentication information and the user data stored on theauthentication server or associated database, as well as if there is amatch between the voice audio data and the training data. If there is amatch, the process 2600 moves to step 2616 and grants the user access tothe voice authenticated item. The process 2600 then ends at end block2618. If at decision block 2614 no match is found, the process 2600 endsat end block 2618.

Referring now to FIG. 27 , there is illustrated an account creationabuse prevention process 2700, in accordance with various embodiments ofthe present disclosure. At step 2702, a user attempts to create a newuser account with a service provider, such as a website or other serviceprovided to users such as a mobile device app, etc., that allows foraccounts to be created. At step 2704, voice authentication setup isinitiated. The voice authentication setup may be required to completeaccount creation. At step 2706, voice capture is activated and arecording of the user's voice is captured and stored as a voice audiofile. At step 2708, the created voice audio file is encoded with otheruser information into a CHM file, as described herein. The other userinformation may include information the user has entered to be used forthe user's potential new account, such as usernames, passwords, personalinformation, security questions and answers, etc. The other userinformation may be in a text data type of format. At step 2710, theencoded CHM file is transmitted to an authentication server.

At step 2712, the authentication server decodes the CHM file to separatethe voice audio data stream from the other user information data stream.At step 2714, the authentication server compares the voice audio datawith voice training data on an associated database and may also performspeech recognition processes using an audio conversion engine. Also, atstep 2714, the authentication server may compare the other userinformation data with user data stored on the authentication server orassociated database. At decision block 2716, it is determined whetherthe data decoded from the CHM file matches data stored on theauthentication server or the associated database for a user that waspreviously banned from the service. All the textual user information maybe checked against user data stored on the authentication server or thedatabase for a match.

In some cases, if the user is attempting to create an account afteranother account owned by the user was banned, the user may use falsetextual data to try to create a new account. Therefore, theauthentication server may also compare the voice data decoded from theCHM file against voice data stored on the authentication server or thedatabase to determine if the user's voice data is already present on thedatabase. If no match is found, at step 2718, the user account may becreated, barring no other issues. If at decision block 2716 a match isfound, at step 2720, account creation may be denied to prevent the userfrom abusing account creation to circumvent a ban on the user's account.

Referring now to FIG. 28 , there is illustrated a voice-to-text andindexing process 2800 utilizing CHM files, in accordance with variousembodiments of the present disclosure. The process 2800 begins at step2802 where a device, such as authentication server 2502, decodes anaudio data stream from a CHM file. At step 2804, the audio data streamis compared to stored training data. At step 2806, the audio data streamis converted to textual data, such as by the audio conversion engine2504, and the textual data is stored. At step 2808, the stored textualdata is parsed for keywords, while ignoring common words. For example,the system may be configured to recognize verbs and nouns, and ignorearticle adjectives, conjunctions, etc.

At decision block 2810, it is determined whether a keyword is foundduring the text parsing operation of step 2808. If not, the process 2800ends at end block 2818. If so, the process 2800 flows to step 2812. Atstep 2812, each instance of the particular keyword found during theparsing operation is counted. At decision block 2814, it is determinedwhether the number of instances counted for the keyword exceeds athreshold. In some embodiments, the number of times a keyword appearsmay be used in determining whether to index that keyword. For example,if the word is only used one time, the word may not be indexed. However,if the word is used over ten times, for example, the word may beindexed. If at decision block 2814 it is determined that the number ofcounted instances for the keyword does not exceed the threshold, theprocess 2800 moves back to decision block 2810 to determine if anotherkeyword is found. If at decision block 2814 it is determined that thenumber of counted instances for the keyword exceeds the threshold, theprocess moves to step 2816. At step 2816, the keyword and the instancesof the keywords are indexed. The process then moves back to decisionblock 2810 to determine if additional keywords are found during theparsing operation. If not, the process 2800 ends at end block 2818.

Referring now to FIG. 29 , there is illustrated a diagrammatic view of adatabase keyword index system 2900, in accordance with variousembodiments of the present disclosure. There is illustrated a database2902 having stored thereon keyword data 2904. The keyword data 2904 maybe user-specific, or may encompass all users who provided text to thedatabase 2902. The keyword data may include keywords 2906. The keywords2906 may be all keywords previously indexed. The keyword data 2904 mayalso include instance data 2908 that includes the number of instances akeyword was used. Location data 2910 may also be stored. The locationdata 2910 may indicate where the text data that the keyword appears isstored in association with the database, such as a server. The locationdata 2910 may also be configured to point to each specific instance ofthe keyword. For example, if a keyword appears five times in aparticular document, the location data 2910 may provide a file path tothat document, and a link to each instance of the keyword within thatdocument.

Referring now to FIG. 30 , there is illustrated a keyword search process3000, in accordance with various embodiments of the present disclosure.The process 3000 begins at step 3002 where a search operation isinitiated on stored text. At step 3004, one or more search keywords arereceived by a device, such as a server. At decision block 3006, it isdetermined whether any instances of the one or more search keywords arefound. If not, the process 3000 ends at end block 3014. If so, theprocess 3000 moves to step 3008. At step 3008, the located keywords areranked based on the number of instances for that keyword. At step 3010,the instances of the found keywords may be presented to a user in a listordered by rank. At step 3012, in response to the user selecting one ofthe presented instances of the found keywords, contextual text or thefull text document in which the keyword appears may be presented to theuser. The process 3000 then ends at end block 3014.

Referring now to FIG. 31 , there is illustrated a diagrammatic view of acombined file encoding process 3100 with remainders. There isillustrated a first file 3102, and second file 3104, and a combined file3106. It is determined in accordance with processes described hereinthat the byte block size for the first file is 3 bytes, while the byteblock size for the second file is 2 bytes. Either before encoding thebytes to the combined file 3106, or after such encoding, a headersection 3108 of the combined file 3106 may be populated with informationpertaining to the first file 3102 and the second file 3104. Thisinformation may include the byte block size assigned to the first file3102 and the second file 3104. For example, as shown in FIG. 31 , theheader section 3108 includes the information “F1BBS(3); F2BBS(2);F1BBSR(1),” which indicates that the first file byte block size is equalto 3, the second file byte block size is equal to 2, and the first filehas a remainder of bytes equal to 1 byte.

As illustrated in FIG. 31 , the combined file 3106 has a body section3110 including a plurality of bytes. Since the byte block size for thefirst file 3102 is three, at a first step the first three bytes of thefirst file 3102 are written to the combined file 3106. Since the byteblock size for the second file 3104 is two, at a second step the firsttwo bytes from the second file 3104 are written to the combined file3106 after the first three bytes written to the combined file 3106 fromthe first file 3102 in the first step. At a third step, the next threebytes of the first file 3102 are written to the combined file 3106 afterthe first two bytes written to the combined file from the second file3104. At a fourth step, the next two bytes are written from the secondfile 3104 to the combined file 3106. At a fifth step, the next threebytes are written from the first file 3102 to the combined file 3106. Ata sixth step, the next, and last, two bytes are written from the second3104 file to the combined file 3106.

At a seventh step, the first file only has one byte left, which issmaller than the assigned three byte block size. The encoder may havealready analyzed the first file 3102 and determined there would be onebyte left, or the encoder may have set the byte block size for the firstfile 3102 and, when encountering the end of the file with a number ofbytes less than the byte block size, the encoder simply takes theremaining bytes and writes them to the combined file 3106. At an eighthstep, the second file 3104 may be checked again, and it may bedetermined that the second file 3104 does not have any bytes left. Inthis case the encoder may simply move on, or it may write a NULL valuein the combined file 3106.

Referring now to FIG. 32 , there is illustrated a diagrammatic view of acombined file decoding process 3200. The process 3200 includes anapplication processing block 3202 that may be part of the codec ordecoder, or part of another application, such as an application thatcalls a codec or an encoder/decoder API in order to perform encoding ordecoding steps. A combined file 3204 is decoded to extract data streamspertaining to a first file 3206 and a second file 3208. During thedecoding process, the application processing block 3202 may receive datastreams pertaining to the first file 3206 and the second file 3208 abyte block at a time. As the application processing block 3202 receiveseach byte block, the application processing block 3202 may write thebyte block to the appropriate one of the first file 3206 or the secondfile 3208. At substantially the same time, the application processingblock 3202 may also utilize the byte blocks for further processing orfor displaying content included within the byte blocks.

For example, as illustrated in FIG. 32 , at a first step 1 a byte blockthe size of three bytes is copied from the combined file 3204 by theapplication processing block 3202. At a second step 2, the applicationprocessing block writes to the first file 3206 the byte block copied inthe first step 1. At substantially the same time as the second step 2, astep 2′ is taken to provide the bytes copied in the first step 1 to acontent viewer or player 3210, to display the content copied from thecombined file 3204 so far. For example, if the content viewer or player3210 allows for audio to be played back, and the bytes copied in thefirst step 1 pertain to audio content, the content viewer or player 3210may begin playing the partial audio file copied in the first three bytesof the combined file 3204. Thus, since streams of data are being pulledfrom the combined file 3204, this data can be used even before all thedata byte are copied from the combined file 3204 and written to thefirst file 3206 and the second file 3208.

At a third step 3, the application processing block 3202 copies the nexttwo bytes from the combined file 3204 that pertain to the second file3208. At a fourth step, the application processing block 3202 writes thetwo bytes copied at the third step 3 to the second file 3208. Atsubstantially the same time, at a step 4′, the application processingblock 3202 may provide the two bytes copied in the third step 3 to thecontent viewer or player 3210 so that the content viewer or player 3210may begin using the data stream pertaining to the second file 3208. Thispattern may continue until all the bytes from the combined file havebeen copied and written to the first file 3206 and the second file 3208.

FIG. 33 illustrates a browser window 3300 showing a webpage 3302 thatincludes a call 3304 to a decoder script according to variousembodiments of the present disclosure. The webpage 3302 further includesan image 3306 with accompanying audio 3308 presented on the webpage3302. As described with respect to FIG. 12 , a CHM, or CHIF, file may bereceived by a device on which a browser or other means of accessing webcontent resides. In some embodiments, the call 3304 to the decoderscript initiates a decoding process by the script. In some embodiments,the script can be in JavaScript or another scripting language.

In some embodiments, the decoder script is stored as a component of thewebpage 3302, and the call 3304 to the script stored on the webpage 3302causes the script to decode the CHIF file that includes the image 3306and the audio 3308, to both display the image 3306 and playback theaudio 3308 in the webpage 3302. In some embodiments, the call 3304 tothe decoder script transmits a request to a server to decode the CHIFfile and then transmit the separated content to the device on which thewebpage is loaded, the separated content providing the content to bepresented on the webpage 3302. Once the webpage 3302 is loaded in thebrowser window 3300 the image 3306 and other webpage content such asuser interaction buttons 3310 and textual product information 3312 maybe presented. The audio 3308 may begin playing once the website isloaded, or the audio 3308 may only play upon a trigger instructing theaudio 3308 to play. For example, the audio 3308 may only play when amouse cursor 3314 is placed over the image 3306, when the image 3306 isclicked on, or when the webpage 3302 is scrolled such that the image3306 is viewable in the browser window 3300, with the audio 3308stopping once the user scrolls away from the image 3306.

Block 3316 illustrates a function of the decoder script. Block 3316illustrates that the decoder script transmits CHIF access information tothe server. The CHIF access information can include various dataconcerning the CHIF file embedded in the webpage, the user or userdevice that initiated the decoding of the CHIF file, the webpage 3302 orowner of the webpage 3302, or other data. For example, the decoderscript can be configured to transmit to the server informationconcerning the identity of the particular user that navigated to thewebpage and viewed the CHIF file content, device data of that particularuser's electronic device used to view the webpage 3302, informationregarding the nature of the CHIF file, or other information. Forexample, an identity of the user can be transmitted to the server todetermine if a user who should not have access to the CHIF file is ableto access the file, or for other identification reasons. As anotherexample, the device data can be transmitted to the server to determinethe types of devices being used to view CHIF content, in order to managedevice compatibility.

As yet another example, the owner of the webpage 3302, the webpage URL,or other information pertaining to the webpage 3302 or the owner can betransmitted to the server to track where CHIF files are being used anddecoded. In some embodiments, the CHIF decoder can be proprietary,requiring users who use encoded CHIF files, such as embedding an encodedCHIF file on a webpage, to call the decoding function from the server inorder to allow for the decoding of the CHIF files. CHIF file userinformation, such as the URL or the owner of the webpage 3302, can betransmitted to the server to provide information on who is using encodedCHIF files. In some embodiments, unauthorized usage, such as by theowner of the webpage 3302, the user of the device loading and viewingthe webpage 3302, or other sources, can be blocked by the server. Forexample, when the decoder script is called, the server can determine theentity calling the script, determine if access to use the decoder isauthorized, and, if not, block or deny access to, or use of, the CHIFdecoder. It will be understood that the CHIF file described with respectto FIG. 33 can include any number of content types, such as text files,audio files, image files, video files, or other file types. In someembodiments, the script can also capture information concerning the CHIFfile, such as an identifier assigned to the CHIF file upon creation ofthe CHIF file, as well as metadata of the CHIF file.

FIG. 34 illustrates a flowchart of a CHIF file decoding and blockingprocess 3400 in accordance with various embodiments of the presentdisclosure. The process 3400 begins at block 3402. At block 3402, a CHIFfile is embedded in a webpage, and stored in a location, for subsequentexecution on the webpage. It will be understood that the process 3400 isnot limited to embedding CHIF files on webpages and decoding theembedded CHIF file. The process 3400 is applicable to any scenario inwhich a CHIF file is stored and decoded using a decoder script. Forexample, a medical office may store a CHIF file including patientdocuments, and may call a decoder script from a remote server to decodethe CHIF file to view the patient documents.

At block 3404, a user triggers a CHIF file action, such as, in the casein which a CHIF file is embedded in a webpage, placing a mouse cursorover an area to display CHIF file contents. At block 3406, the webpagecalls a script for decoding the CHIF file. In other embodiments, adecoder script may be otherwise called, such as when a CHIF file storedby an office is loaded into a decoder application that calls a scriptfor remote decoding of the CHIF file. At block 3408, the scripttransmits CHIF file access information to the server. As described withrespect to FIG. 33 , the CHIF file access information can includevarious data concerning the CHIF file, the user or user device thatinitiated the decoding of the CHIF file, a webpage or owner of a webpagein which a CHIF file is embedded, an owner of a storage system in whicha CHIF file is stored, or other data.

For example, the script can be configured to transmit to the serverinformation concerning the identity of the particular user thatnavigated to the webpage and viewed the CHIF file content, device dataof that particular user's electronic device used to view the webpage, orother user or device information. For example, an identity of the usercan be transmitted to the server to determine if a user who should nothave access to the CHIF file is able to access the file, or for otheridentification reasons. As another example, the device data can betransmitted to the server to determine the types of devices being usedto view CHIF content, in order to manage device compatibility. As yetanother example, the owner of the webpage, the webpage URL, or otherinformation pertaining to the webpage, or the owner of the storagelocation of the CHIF file, can be transmitted to the server to trackwhere CHIF files are being used and decoded. In some embodiments, theCHIF decoder can be proprietary, requiring users who use encoded CHIFfiles, such as embedding an encoded CHIF file on a webpage, to call thedecoding function from the server in order to allow for the decoding ofthe CHIF files. CHIF file user information, such as the URL or the ownerof the webpage, can be transmitted to the server to provide informationon who is using encoded CHIF files.

At block 3410, the server stores the CHIF file access information. Atdecision block 3412, the server determines whether to block decoding ofthe CHIF file. This determination can be based on the variousinformation of the CHIF file access information described above, such asan identity of a user attempting to decode a CHIF file. The server candeny or block decoding of a CHIF file if a particular user isunauthorized to use the decoding function. If the server determines thatdecoding of the CHIF file should be blocked, the process 3400 moves toblock 3414. At block 3414, the server instructs the script, anotherscript, or a process, to block CHIF file access for the current decodingattempt. At block 3416, the script blocks the decoding of the CHIF fileand presents an error message to the user. In some embodiments in whicha CHIF file is embedded in a webpage, the area of the webpage wherecontent of the CHIF file would appear may display the error message, orsome other icon indicating that decoding is blocked. In someembodiments, a window or other area can be displayed with an errormessage or icon. The process then ends at block 3420. If at decisionblock 3412 the server determines that decoding is not to be blocked, theprocess moves to block 3418. At block 3418, the script executes thedecoding process to decode the CHIF file for presentation of thecontents of the CHIF file to the user. The process then ends at block3420.

FIG. 35 illustrates a CHIF encoding and decoding system 3500 inaccordance with various embodiments of the present disclosure. EncodedCHIF files can be stored remotely for retrieval by an organization, orbetween organizations that share CHIF files. As illustrated in FIG. 35 ,the system 3500 includes a remote storage 3502. The remote storage 3502can include one or more servers 3504, a cloud storage system 3506,and/or a storage archive 3508, to store CHIF encoded CHIF files forlater retrieval and decoding. The remote storage 3502 can also include abackup storage location or backup functions 3509, to backup informationstored on the remote storage 3502 in the case of a hardware or otherstorage failure. A CHIF encoder 3510 stored, or accessible and used, byan organization receives data from a plurality of sources, encodes thedata into a CHIF file as described in the various embodiments herein,and transmits CHIF file data 3512 to the remote storage 3502. The datafrom the plurality of sources can include various information that theorganization wishes to include in a CHIF file, such as files that are ofdifferent types, but provide complementary information.

For example, if an organization is a medical organization, the data fromthe plurality of sources can include information from a digital imagingand communications (DICOM) system. DICOM systems provide a standardizedsystem for handling, storing, printing, and transmitting informationrelated to medical imaging or radiology, such as X-Ray, magneticresonance imaging (MRI) data, and other imaging modality data. DICOMsystems often include file format definitions and a networkcommunications protocol, such as TCP/IP, to communicate between DICOMsystems. For instance, as illustrated in FIG. 35 , first image/videodata and metadata 3514 can be extracted from a DICOM system and providedto the CHIF encoder 3510. The first image/video data and metadata 3514can include patient radiology image/video data and its associatedmetadata, and other patient data. Additionally, second image/video dataand metadata 3516 can be extracted from DICOM imaging modalities 3518,such as via a DICOM protocol, and provided to the CHIF encoder 3510, asillustrated in FIG. 35 . DICOM imaging modalities can include dataobjects that include a number of attributes, such as a name, anidentifier (ID), image pixel data, DICOM modalities, and otherinformation. DICOM modalities can include various information orattributes known to those skilled in the art that pertain to, anddesignate, the types of DICOM information, such as autorefraction (AR),content assessment results (ASMT), audio (AU), document (DOC), digitalradiography (DX), electrocardiography (ECG), endoscopy (ES), panoramicX-Ray (PX), slide microscopy (SM), and many others. The CHIF encoder3510 can also be provided with reports 3520, such as doctor audiodictations, dictation transcriptions, written reports, annotations, labreports, radiology reports, patient information documents, and any otherinformation the medical organization wishes to include in the CHIF file.

The information the medical organization wishes to include in the CHIFfile, such as the data 3514 and 3516, and the reports 3520, are providedto the CHIF encoder 3510. The CHIF encoder 3510 combines thisinformation into a single encoded CHIF file, using processes asdescribed in the various embodiments of this disclosure. The single CHIFfile thus can contain various information that are related and relevantfor a patient. In traditional medical systems such as DICOM systems,various files pertaining to a patient may be stored at differentlocations in storage, and may not be obviously related or stored in sucha way that a medical professional can review all the information intandem. For example, imaging files may be stored in one location orsystem, while doctors' notes are stored in another location or system,doctor dictations in another location or system, lab reports in anotherlocation or system, etc. Encoding the data into a single CHIF fileallows for organizations to associate all the data provided to the CHIFencoder 3510, and this data can be presented and reviewed together byprofessionals, allowing for professionals to have all relevantinformation before them in the CHIF file. It will be understood theresulting CHIF file can thus include data from various data types, suchas image data, audio data, text data, radiology system data, video data,or other data types.

In some embodiments, files and data provided to the CHIF encoder 3510can be indexed based on keywords or other parameters, and an index filecreated, to provide for the searchability of CHIF files either on alocal storage or the remote storage 3502. In some embodiments, the CHIFfile can be transmitted to the remote storage 3502, and the remotestorage can decode the CHIF file, create the index, and re-encode theCHIF file for storage. The created index file includes data parsed fromthe CHIF file, and allows for organizations with access to the remotestorage 3502 to search for CHIF files pertaining to certain criteria,such as searching for all CHIF files associated with a patient,associated with a doctor, pertaining to a particular medical condition,or other search parameters.

Additionally, during an imaging session such as an X-Ray or MRI session,there may be dozens or even hundreds of images recorded. However, adoctor or other qualified medical professional may only want to focus ona certain few relevant images in studying a patient complaint, and mayonly refer to these certain images in a related report, audio or textualdictation, notes, or other document types. Instead of a medicalprofessional being required to sift through all images stored in asystem, such as a DICOM system, only these certain images used inunderstanding the patient's complaint or status can be provided to theCHIF encoder 3510 along with the accompanying reports, notes,dictations, etc., such that, upon later review of the contents of theCHIF file, the CHIF file provides only the relevant information used inreaching a conclusion or diagnosis of the patient. Additionally,reducing the number of images or other content to include in the CHIFfile provides for a smaller-sized CHIF file that can be more easilytransmitted within or across organizations.

DICOM files can be exchanged between two entities that are capable ofreceiving image and patient data in DICOM format. A medical organizationusing a DICOM or other medical system or standard can leverage the CHIFencoding and decoding technology described herein to combine multiplesources of patient and diagnostic information into a single CHIF filefor storing at the remote storage 3502, or at a local storage location.The medical organization can then retrieve the CHIF file from storagewhen review of the patient and diagnostic information is to beperformed, or other medical organizations can be granted access to theCHIF file, such as when a patient is referred to a specialist. Forexample, a first medical organization can encode, using the CHIF encoder3510, various data into a single CHIF file, and transmit the CHIF filedata 3512 to the remote storage 3502. Once stored at the remote storage3502, the CHIF files can be retrieved by various authorizedorganizations, such as by a CHIF-enabled system 3522 or a not fullyCHIF-enabled system 3524.

The CHIF-enabled system 3522 is a system that is configured to retrieveCHIF files from the remote storage 3502, and is configured with one ormore applications that enable the presentation of CHIF file contentssimultaneously, such as within the same application window. For example,the CHIF-enabled system 3522 can include the one or more applicationsfor viewing CHIF content installed on various workstations, such asradiology workstations, in hospital information systems (HIS),electronic medical records (EMR) systems, radiology information systems(RIS), or other systems, workstations, computers, or mobile devices usedby the organization. The one or more applications for viewing CHIFcontent simultaneously can decode the CHIF file into its various datatypes 3526, such as images, metadata, reports, text content, audiocontent, and other data, and present the content to users in theCHIF-enabled system 3522, such as that shown in the example in FIG. 24herein. CHIF-enabled systems 3522 can thus provide for the review of allrelevant information together, such as patient demographic, diagnostic,and radiology data. In some embodiments, CHIF-enabled systems 3522 canalso store CHIF files instead of decoding the CHIF file into its variouscomponents and saving them individually. This allows for the CHIF filesto be the main file type for content stored on the CHIF-enabled systems3522, and the one or more applications for viewing the CHIF contentprovides a central system for viewing content. In some embodiments, CHIFfiles may not be stored locally, but are stored on the remote storage3502 and accessed temporarily for viewing.

The not fully CHIF-enabled system 3524 may not include the one or moreapplications for presenting CHIF content simultaneously. The not fullyCHIF-enabled system 3524 requests and retrieves CHIF file data 3528 fromthe remote storage 3502, and views the information after decoding by aCHIF decoder 3530. In some embodiments, the CHIF decoder 3530 is storedlocally at the not fully CHIF-enabled system 3524, and the CHIF decoder3530 decodes the CHIF file into its various original data components forstorage in appropriate systems, such as storing images and metadata inDICOM or other archiving systems, reports in EMR/HIS systems, convertingreports into PDF or faxing reports, storing text content in RIS systems,etc. In some embodiments, legacy data 3532, such as data from RIS, EMR,HIS, and picture archiving and communication (PAC) systems, can begradually encoded into CHIF files to begin converting not fullyCHIF-enabled systems 3524 into CHIF-enabled systems.

In some embodiments, a CHIF-enabled viewing portal 3534 can be used bythe not fully CHIF-enabled systems 3524. The CHIF-enabled viewing portal3534 in some embodiments is a solution or application hosted by theremote storage 3502 or other remote system. The CHIF-enabled viewingportal 3534 allows for not fully CHIF-enabled systems 3524 to view CHIFfile content simultaneously much like the one or more applications ofthe CHIF-enabled systems 3522. The CHIF-enabled viewing portal 3534running at the remote or cloud-based location decodes, or receives froma decoder, CHIF file content and presents the CHIF file datasimultaneously, or at least provides tools or options for viewing thevarious CHIF file data, such as switching between views of the variousdata. At the not fully CHIF-enabled system 3524, users can view thecontent presented by the CHIF-enabled viewing portal 3534 in aclient-side application such as a web browser or dedicated applicationfor viewing the CHIF content presented on various devices or mediums3536 such as on a monitor, tablet device or other mobile device, orloaded from an external storage device such as a compact disc (CD),digital versatile disc (DVD), universal serial bus (USB) storage, orother external storage devices. The CHIF-enabled viewing portal 3534 canpresent content to the not fully CHIF-enabled systems 3524 without theCHIF files being downloaded to the not fully CHIF-enabled system 3524.In some embodiments, the CHIF-enabled systems 3522 can also use theCHIF-enabled viewing portal 3534, either in lieu of, or in conjunctionwith, the one or more applications for viewing CHIF content. Asillustrated in FIG. 35 , the system 3500 provides for the creation,storage, sharing, and decoding of CHIF files across a plurality oforganizations, with the CHIF files providing multiple forms of relateddata together for review.

FIG. 36 illustrates a CHIF encoding and decoding process 3600 inaccordance with various embodiments of the present disclosure. Theprocess 3600 can be used with a CHIF encoding and decoding system, suchas the system 3500. A CHIF encoding process 3602, such as that executedby a CHIF encoder application, receives various forms of data andencodes the data into CHIF data 3604. The various forms of data forencoding can include report data 3606 from a networked system, and textcontent 3608 from one or more dictations. The report data 3606 from thenetworked system can include lab/pathology report data 3610 from thenetworked system, diagnostic imaging report data 3612 from one or moredictations, diagnostic imaging report data 3614 from storage, and otherinformation. The CHIF encoding process 3602 can also receive informationsuch as DICOM system information. The DICOM system information caninclude image data 3616 and metadata 3618, extracted during anextraction process 3620 from a DICOM system. The extraction process 3620can extract image/video data and metadata from various DICOM systemsources, such as DICOM data from an archiving storage system 3622, suchas a PAC system or a vendor neutral archive (VNA) system, DICOM datafrom imaging modalities 3624, and/or DICOM data from CD/DVD files 3626,or other storage types. The CHIF encoding process 3602 can also encode,into a CHIF file, audio data 3628 from a dictation system, audio data3630 from a microphone input, and/or audio data 3632 from storage. Itwill be understood that other data can be provided to the CHIF encodingprocess 3602 as desired for inclusion in a resulting CHIF file.

Once the CHIF encoding process 3602 combines all input data as disclosedin the various embodiments herein, the resulting CHIF data 3604 of thecreated CHIF file is transmitted to a storage location 3634 for storingat the storage location 3634. In some embodiments, the storage locationcan be the remote storage 3502. In some embodiments, an index filecreated from the content sources before or during the encoding process3602 can also be transmitted to the storage location 3634. The storagelocation 3634 can include an associated database 3636 that keeps arecord of CHIF files stored at the storage location 3634. For example,each CHIF file can have a universally unique identifier (UUID) added tothe metadata of the CHIF file upon creation of the CHIF file, and thedatabase 3636 can record each UUID in association with the storagelocation of the CHIF files on the storage location 3634. In someembodiments, the metadata and/or the UUID of the CHIF file can include atimestamp or other further identifying information. In some embodiments,the database 3636 can also store index information for indexesassociated with CHIF files, to provide for keyword or other parametersearching of the indexes for retrieval of CHIF files matching, or beingrelated to, the keywords or other parameters.

CHIF files stored at the storage location 3634 can be requested by aCHIF decoding process 3638 to decode the CHIF files for viewing thecontent of the CHIF files. For example, in some embodiments, anorganization can create a CHIF file using the CHIF encoding process3602, store the CHIF file at the storage location 3634, and laterretrieve the CHIF file from the storage location 3634 to decode the CHIFfile by the CHIF decoding process 3638. In some embodiments, a firstorganization creates and stores the CHIF file, while a secondorganization retrieves the CHIF file for decoding by the decodingprocess 3638. The decoding process 3638 decodes the various datapreviously encoded into the CHIF file, as described with respect to thesystem and processes herein. For example, decoding the CHIF data 3604encoded as illustrated in FIG. 36 results in extracting from the CHIFdata 3604 the report data 3606 from the networked system, including thelab/pathology report data 3610 from the networked system, the diagnosticimaging report data 3612 from one or more dictations, the diagnosticimaging report data 3614 from storage, and other information. The textcontent 3608 from one or more dictations can also be extracted from theCHIF data 3604.

The CHIF decoding process 3638 can also extract from the CHIF data 3604the image data 3616 and metadata 3618. The image data 3616 and themetadata 3618 can then be provided to a formatting process 3640 thatformats the images/video data and metadata from the image data 3616 andthe metadata 3618 into DICOM compatible data for storage in a DICOMsystem. This compatible data can be formatted as DICOM data for anarchiving storage system 3642, such as a PAC system or a VNA system,DICOM data for imaging modalities 3644, and/or DICOM data for CD/DVDfiles 3646, or other storage types. The CHIF decoding process 3638 canalso decode CHIF data 3604 from the CHIF file audio data 3628 from thedictation system, the audio data 3630 from the microphone input, and/orthe audio data 3632 from storage. It will be understood that other datacan be decoded by the CHIF decoding process 3638. The process 3600provides for a plurality of related data, such as that illustrated inFIG. 36 , to be efficiently stored and shared across a plurality ofplatforms and systems.

FIG. 37 illustrates a combined file storage and retrieval process 3700in accordance with various embodiments of the present disclosure. Theprocess 3700 can be used with any of the systems and other processesdescribed herein, such as with system 3500 and process 3600. The process3700 can be performed and executed by a processor such as the processor6502 described herein. The process 3700 begins at block 3702. At block3702, input data from a plurality of data types is input into a CHIFencoder. At block 3704, the CHIF encoder combines the data from theplurality of data types into a single CHIF file and assigns a UUID tothe CHIF file. At block 3706, the CHIF file is transmitted to a serverfor storage. At block 3708, the server designates authorized partiesthat can access the CHIF file and stores access rights related to theauthorized parties and the CHIF file in a database. For example, theserver can store a plurality of user IDs in a database in associationwith one or more CHIF files, designating that the IDs have access to theone or more CHIF files. The users that are to have access rights to theCHIF files can, in some embodiments, be included in a message to theserver sent with, or including, the CHIF file to be stored.

At block 3710, the server receives a request for access to the storedCHIF file. In some embodiments, only certain users within anorganization may be granted access to certain CHIF files, to ensure thatthe content of CHIF files cannot be accessed by anyone at theorganization. In some embodiments, a first organization can grant CHIFfile access to another organization, and/or one or more users withinthat other organization, and the server thus only allows for the otherorganization to access the specific CHIF file, even though multipleother organizations may have access to the server. At decision block3712, the server determines whether the requesting party is authorizedto access the stored CHIF file. If the server determines that therequesting party is not authorized to access the stored CHIF file, theprocess 3700 moves to block 3714. At block 3714, access to the CHIF fileis denied, and the process 3700 returns to block 3710. If at decisionblock 3712 the server determines that the requesting party is authorizedto access the CHIF file, the process moves to block 3716.

At block 3716, the server transmits the CHIF file to the requestingparty. At block 3718, a device associated with the requesting partyinputs the CHIF file into a decoder. In some embodiments, the decodingcan be performed at the server and the decoded CHIF file contents aretransmitted to the requesting party or otherwise displayed to therequesting party, such as in the CHIF-enabled viewing portal 3534. Atblock 3720, the CHIF decoder separates the CHIF file data, as describedwith respect to the systems and processes disclosed herein, into data ofthe plurality of data types used to create the CHIF file by the encoderat block 3704. At block 3722, the separated content and/or data ispresented to the authorized party. The process 3700 ends at block 3724.It will be understood that in various embodiments the server can beother devices such as local devices that determine CHIF file access. Itwill also be understood that the transmission of data and CHIF files inprocess 3700 and the other systems and processes disclosed herein can besecured via encryption and/or other security processes and protocols.

FIG. 38 illustrates a combined file system compatibility determinationprocess 3800 in accordance with various embodiments of the presentdisclosure. The process 3800 can be used with any of the systems andother processes described herein, such as with system 3500 and process3600. The process 3800 can be performed and executed by a processor suchas the processor 6502 described herein. The process 3800 begins at block3802. At block 3802, a server stores a created CHIF file. At block 3804,the server receives an access request from an authorized party. Atdecision block 3806, the server determines if the access requestincludes a request to use a viewing portal to view the content of therequested CHIF file, such as the CHIF-enabled viewing portal 3534. If atdecision block 3806 the server determines that the party is requestinguse of the viewing portal, the process 3800 moves to block 3808. Atblock 3808, a decoder stored at the server decodes the requested CHIFfile. At block 3810, the server executes the CHIF viewing portalapplication, loads the decoded content of the CHIF file in the CHIFviewing portal application, and transmits a display of the content tothe application stored by the party. The CHIF viewing portal applicationthus provides a cloud-based software application for providing viewingof CHIF files without software needing to be installed locally by therequesting party.

If at decision block 3806 the server determines that the party is notrequesting use of the viewing portal, the process 3800 moves to block3812. At block 3812, the server transmits the CHIF file to theauthorized party. At decision block 3814, an authorized party devicedetermines whether the authorized party, or a system associated with theauthorized party, is CHIF-enabled. If so, at block 3816, a decoder usedby the authorized party decodes the CHIF file and presents the decodedcontent simultaneously in a viewer application, as described withrespect to various embodiments disclosed herein. In some embodiments, asingle application can include both decoding and content viewingfunctions. The process 3800 then ends at block 3820. If at decisionblock 3814 the authorized party device determines that the authorizedparty is not fully CHIF-enabled, the process 3800 moves to block 3818.At block 3818, the decoder of the authorized party decodes the CHIF fileand provides the decoded content to various relevant systems, such aspatient record systems, imaging or DICOM systems, or other systems. Theprocess 3800 then ends at block 3820.

FIG. 39 illustrates a combined file signing and locking process 3900 inaccordance with various embodiments of the present disclosure. Theprocess 3900 can be used with any of the systems and other processesdescribed herein, such as with system 3500 and process 3600. The process3900 can be performed and executed by a processor such as the processor6502 described herein. The process 3900 begins at block 3902. At block3902, an encoder receives a plurality of data for encoding. At block3904, the encoder combines the data into a combined CHIF file. Atdecision block 3906, it is determined whether an attempt to includeadditional data into the CHIF file is detected. In some embodiments,even after creation of a CHIF file, additional content can be combinedinto the CHIF file by the encoder, and the metadata is updated toreflect the additional content. In some embodiments, to create a newCHIF file, the original CHIF file is decoded, and then a new CHIF fileis created by encoding the decoded content and the new content. Forexample, a doctor may receive a CHIF file that includes radiologyimages, lab reports, and other information to be used by the doctor indiagnosing or treating a patient. The doctor can review thisinformation, and then provide additional information to be included inthe CHIF file, such as images, audio dictation, textual notes, reports,or other information.

If at decision block 3906 it is determined that additional data is to beincluded in the CHIF file, the process 3900 returns to block 3904 tocombine the additional data with the original data into a single CHIFfile. If at decision block 3906 no additional data is to be combinedwith the CHIF file, the process 3900 moves to block 3908. At block 3908,the processor requests an authorized signature. In some embodiments, anauthorized signature can be applied once contents of a CHIF file havebeen reviewed, potential additional content is added, and the contentsof the CHIF file are finalized. For example, if a reviewing doctorreviews all original information in the CHIF file, and providesadditional data in the form of a diagnosis or other conclusions, thedata can be combined into a modified or new CHIF file, and the doctorcan append an authorized doctor's signature to the file, indicating acomplete diagnosis or opinion from the doctor. In some embodiments, thedoctor can also remove delete contents not relied upon in studying thepatient, such as extra radiology images not relied upon, and the CHIFfile can be re-encoded with the relevant content and the doctor'ssignature. At block 3910, the processor appends the authorized signatureto the metadata of the CHIF file, triggering a locked state in whichediting of the CHIF file is locked, restricted, or prohibited. In someembodiments, content to be included in the CHIF file such as radiologyimages and lab reports can be reviewed before they are encoded into aCHIF file, and the doctor's signature is included and the CHIF filelocked upon initial encoding of the CHIF file. Locking the CHIF fileserves to provide that the CHIF file can no longer have additionalinformation added to the CHIF file. This provides that CHIF files markedas complete with an authorized signature cannot be tampered with orotherwise altered. For example, although a second doctor may view a CHIFfile that includes a first doctor's opinion and the underlyinginformation, the second doctor, if desiring to create a CHIF file withthe second doctor's own opinions, would create another CHIF file with anauthorized signature of the second doctor. The other CHIF file mayinclude a portion or all of the original CHIF file's data and content.

At block 3912, the processor transmits the CHIF file to a storagelocation, such as a server. At block 3914, the server receives a requestfor access to the CHIF file, and, at block 3916, the server transmitsthe CHIF file to the requester. At decision block 3918, a device such asa decoder and/or a processor detects whether the requester attempts toedit the CHIF file. If so, the device generates and displays an errormessage indicating that the CHIF file is locked and not editable. Theprocess 3900 then moves back to decision block 3918, to determine if therequester makes any subsequent edit attempts and, if so, loops throughthe generation and display of the error message at block 3920. If atdecision block 3918 the requester does not attempt to edit the CHIFfile, at block 3922, the CHIF file can be decoded and the contents ofthe CHIF file presented, in accordance with various embodimentsdisclosed herein. The process 3900 ends at block 3924.

FIG. 40 illustrates a combined file tracking process 4000 in accordancewith various embodiments of the present disclosure. The process 4000 canbe used with any of the systems and other processes described herein,such as with system 3500 and process 3600. The process 4000 can beperformed and executed by a processor such as the processor 6502described herein. The process 4000 begins at block 4002. At block 4002,a CHIF encoder receives a plurality of data. At block 4004, the CHIFencoder combines the plurality of data into a single CHIF file. Duringcreation of the CHIF file, the CHIF encoder can create an identifier orUUID for the CHIF file, which can be used to track and search for theCHIF file. In some embodiments, the UUID can be a unique SHA numbercreated by a cryptographic hash function or other process and includedin the metadata of the CHIF file. Since SHA numbers can be usedconsistently across files, the SHA numbers can be recorded and tracked.

At block 4006, the identifier associated with the CHIF file created inblock 4004 is stored in a database in association with a storagelocation of the CHIF file. At decision block 4008, a processordetermines if a request for the CHIF file, or data associated therewith,is received. If not, the process 4000 loops back to decision block 4008.If so, the process 4000 moves to block 4010. At block 4010, theprocessor retrieves the identifier associated with the requested CHIFfile. At block 4012, the processor compares the identifier with thecurrently stored CHIF file, to determine if the identifier stored in thedatabase matches the identifier included in the metadata of the CHIFfile. In some embodiments, mismatched identifiers between the identifierstored in the database and the identifier in the metadata of the CHIFfile can indicate that the CHIF file has been tampered with or altered.

At decision block 4014, the processer determines, based on thecomparison in block 4012, whether the identifier stored in the databaseand the identifier included in the CHIF file match. If so, the processortransmits a message indicating matching identifiers, and can furthertransmit the CHIF file or the CHIF file contents. The process then endsat block 4020. If at decision block 4014 the processor determines thatthe identifiers do not match, the process 4000 moves to block 4018. Atblock 4018, the processor instructs a transmission of a messageindicating that the identifiers are mismatched, alerting one or moreusers of a potential issue with the CHIF file. In some embodiments, theprocessor may block further transmission of the CHIF file until an errorflag relating to the mismatched identifiers is cleared. The process thenends at block 4020.

FIG. 41 illustrates a combined file indexing process 4100 in accordancewith various embodiments of the present disclosure. The process 4100 canbe used with any of the systems and other processes described herein,such as with system 3500 and process 3600. The process 4100 can beperformed and executed by a processor such as the processor 6502described herein. The process 4100 begins at block 4102. At block 4102,a CHIF encoder receives a plurality of data. At block 4104, the CHIFencoder or another process indexes the plurality of data. For example,the CHIF encoder can parse text data included in the plurality of datato look for keywords such as “diagnosis,” “osteoporosis,” “heartdisease,” or any other keywords, or other information such as patientnames and other data. In some embodiments, audio data can be analyzed torecognize keywords, which can be added in text form in the index file.In some embodiments, radiology images can be scanned for attributeinformation such as DICOM attributes, and keywords or other informationrelating to the attributes can be added to the index file. In this way,a searchable index file is provided that allows for CHIF files relatingto certain content, patients, etc., to be searchable on a server orother storage location.

At block 4106, the CHIF encoder combines the data into a CHIF file. Atblock 4108, the CHIF file and its associated index file are stored inassociation at a storage location, such as a server. For example, adatabase accessible by storage location devices or processors can relateindex file storage locations with CHIF file storage locations. When asearch is subsequently performed for keywords included in the indexfile, the CHIF file at the associated storage location can be providedas a search result and retrieved. In some embodiments, a master indexfile can be stored and updated with information pertaining to multipleCHIF files. At block 4110, the processor receives a search request andperforms a search of one or more index files for parameters of thesearch request, such as keywords, patient data, or other data. At block4112, the processor retrieves CHIF files associated with the indexesfound during the search based on the parameters.

At decision block 4114, the processor determines if a combined or totalsize of the retrieved CHIF files exceeds a threshold. For example, thethreshold may be a size value such as 5 gigabytes or any other sizevalue. If the total size does not exceed the threshold, at block 4116the processor creates a file including copies of the retrieved CHIFfiles. As disclosed in the various embodiments herein, binder files canbe created to store multiple CHIF files together. At block 4118, theprocessor transmits the binder file containing the retrieved CHIF filesrelevant to the search to the requester. The process 4100 then ends atblock 4122. If at decision block 4114 the processor determines the totalsize of the retrieved CHIF files exceeds the threshold, the processorcreates a binder file including links to the CHIF files relevant to thesearch, rather than including copies of the CHIF files. The process 4100then moves to block 4118 to transmit the binder file to the requester,and the process 4100 ends at block 4122. The binder file including linksto the CHIF files can then be used by the requester to retrieveindividual CHIF files. The binder file including links thus provides fora binder file with a significantly reduced file size that still providesthe requester with access to the relevant search results. In someembodiments, the binder file can include additional information on theCHIF files to indicate to the requester the contents of the CHIF files.In some embodiments, a graphical user interface (GUI) can be presentedto the requester on a requester's device, listing the search results,information pertaining to the search results, and providing links to theCHIF files.

FIG. 42 illustrates a CHIF file decryption and decoding system 4200 inaccordance with various embodiments of the present disclosure. As alsodescribed herein with respect to FIGS. 33 and 34 , client devicesstoring and using CHIF files remotely from a central CHIF-enabledserver, such as devices viewing a CHIF file in a decoder or viewerapplication, viewing in a web browser a webpage including an embeddedCHIF file, etc., can trigger a script from the server to decode andpresent the contents of the CHIF file. If a user, a CHIF file, awebsite, etc., are considered to be disallowed from decoding the CHIFfile, the decoding of the CHIF file can be blocked by the script.

The system 4200 includes a client device 4202 including a memory 4204.It will be understood that the client device 4202 can be any electronicdevice or computing device, such as that described in FIG. 65 herein.The client device 4202 can store in the memory 4204 an encrypted CHIFfile 4206. In some embodiments, the CHIF file may not be encrypted. Theencrypted CHIF file 4206 can be stored in the memory 4204, for example,when the CHIF file 4206 is created or encoded by the client device 4202,or otherwise retrieved by the client device 4202, such as from a server,from another electronic device, from a webpage loaded in a browser ofthe client device 4202, the contents of the webpage being stored in thememory 4204, or from other sources. As described with respect to thevarious embodiments herein, the encrypted CHIF file 4206 can includemetadata and a body, where the body includes data on the plurality ofcontent or one or more content types encoded into the CHIF file 4206.The metadata can also include a UUID that is unique to the CHIF file4206, and that can be used to track, and associate information with, theCHIF file 4206. In some embodiments, the UUID can be a hexadecimalnumber, such as F122 as shown in FIG. 42 .

When the client device 4202 is to decrypt and decode the encrypted CHIFfile 4206 in order to view the contents of the CHIF file 4206, in someembodiments the client device 4202 does not have access to a decoder,and instead requests decryption and decoding of the encrypted CHIF file4206 from a CHIF server 4208. The client device 4202 transmits adecryption and decoding request 4210 over a network 4212 to the CHIFserver 4208. In some embodiments, the request 4210 includes the UUID forthe encrypted CHIF file 4206, and, in some embodiments, the request 4210includes a copy of the encrypted CHIF file 4206. In some embodiments,the CHIF file 4206 can first be decrypted, and then a request includingthe UUID is sent to the CHIF server 4208.

The CHIF server 4208 has stored thereon or associated therewith a script4214, such as a JavaScript or other script, for decoding and presentingcontents of CHIF files. The CHIF server 4208 also has stored thereon orassociated therewith at least one private key 4216 for decryptingencrypted CHIF files encrypted by a public key. In some embodiments, theprivate key 4216 can be a public key, or some other type of decryptionkey. It will be understood that CHIF files can be encrypted and decodedusing any combination of public or private keys, or any other form forencryption, without deviating from the scope of the present disclosure.The CHIF server 4208 also includes, or is communicatively connected to,a data store or database 4218 that includes a UUID registry 4220 and adomain registry 4222. In some embodiments, the UUID registry 4220 andthe domain registry 4222 can be part of the same registry.

The UUID registry 4220 and the domain registry 4222 are used by the CHIFserver 4208 to determine if a CHIF file, such as the encrypted CHIF file4206, can be decoded and presented. The UUID registry 4220 includes alist of UUIDs that are to be blocked from decoding based on certainparameters, such as if the CHIF file is flagged for inappropriate orgraphic content, such as violence, profanity, etc., if the CHIF fileincludes copyrighted material, if a license for the content contained inthe CHIF file is expired, or for other reasons. During creation of aCHIF file, or at other times, a CHIF file can also be given a daterange, such as a start date and an end date. The start date defines whena CHIF file is allowed to be accessed, decoded, viewed, or otherwiseused by a user, domain, or other entities. The end date defines when theCHIF file can no longer be accessed, decoded, viewed, or otherwise usedby a user, domain, or other entities. Decoding of CHIF files can also beblocked for other reasons, such as based on the identity of theparticular user or client device that sent the request 4210.

The domain registry 4222 is used by the CHIF server 4208 to determine ifwebsites hosting CHIF files are allowed to use the decoder script 4214.For example, the encrypted CHIF file 4206 stored in the memory 4204 ofthe client device 4202 may have been accessed on a website, and therequest 4210 includes a domain name or domain ID for the website onwhich the encrypted CHIF file 4206 is hosted. The CHIF server 4208 canthen look up the domain name or domain ID in the domain registry 4222.If the domain ID is not in the domain registry 4222, the CHIF server4208 can block decoding of the CHIF file. If the domain ID is in thedomain registry 4222, the CHIF server 4208 can review the information inthe domain registry 4222 to determine if decoding of CHIF files hostedon the domain is allowed. If the domain registry 4222 indicates thatdecoding is not allowed, such as if a license for the domain to use CHIFfiles is expired, the CHIF server 4208 can block decoding of the CHIFfile.

If decoding is determined by the CHIF server 4208 to be allowed, eitherindicated by one of or both of the UUID registry 4220 and the domainregistry 4222, the CHIF server 4208 will facilitate decryption anddecoding of the encrypted CHIF file 4206. The CHIF server 4208 thensends a transmission 4224 over the network 4212 to the client device4202 that includes the decryption key 4216 and the CHIF decoder script4214. The client device 4202 uses the decryption key 4216 to perform adecryption 4226 of the encrypted CHIF file 4206 to provide a decryptedCHIF file. The client device 4202 then uses the decoder script 4214 todecode the CHIF file into its separated content components 4228. Theclient device 4202 can then view, playback, or otherwise present, basedon the content types of the separate content components 4228, thecontent components 4228.

FIG. 43 illustrates a CHIF file decryption and decoding process 4300 inaccordance with various embodiments of the present disclosure. Theprocess 4300 can be used with any of the systems and other processesdescribed herein, such as with system 4200. The process 4300 can beperformed and executed by a processor such as the processor 6502described herein. The process 4300 begins at block 4302. At block 4302,a request is received, such as by a processor of the CHIF server 4208,to decrypt and decode a CHIF file stored by the requester, such as theclient device 4202, described with respect to FIG. 42 . At decisionblock 4304, the processor determines if one or more CHIF registries areavailable, such as the UUID registry 4220 and/or the domain registry4222. If the registries are not available, the process 4300 loops backto decision block 4304 until the one or more registries becomeavailable. In some embodiments, the process 4300 can continue on even ifthe registries are not available. If the registries are available, theprocess 4300 moves to decision block 4306. At decision block 4306, theprocessor determines if decoding of the CHIF file is to be blocked,based on information in the registries. For example, registries such asthe UUID registry 4220 and the domain registry 4222 are used todetermine if a CHIF file can be decoded and presented. The UUID registry4220 includes a list of UUIDs that are to be blocked from decoding basedon certain parameters, such as if the CHIF file is flagged forinappropriate or graphic content, such as violence, profanity, etc., ifthe CHIF file includes copyrighted material, if a license for thecontent contained in the CHIF file is expired, or for other reasons.Decoding of CHIF files can also be blocked for other reasons, such asbased on the identity of the particular user or client device that sentthe request in block 4302.

The domain registry 4222 is used to determine if websites hosting CHIFfiles are allowed to use the decoder script 4214. For example, the CHIFfile for which decryption and decoding is requested may have beenaccessed on a website, and the request received in block 4302 caninclude a domain name or domain ID for the website on which theencrypted CHIF file is hosted. The domain name or domain ID can thus belooked up in the domain registry 4222. If the domain ID is not in thedomain registry 4222, the processor can block decoding of the CHIF file.If the domain ID is in the domain registry 4222, the processor canreview the information in the domain registry 4222 to determine ifdecoding of CHIF files hosted on the domain is allowed. If the domainregistry 4222 indicates that decoding is not allowed, such as if alicense for the domain to use CHIF files is expired, the processor canblock decoding of the CHIF file.

If at decision block 4306 the processor determines that decoding of theCHIF file is to be blocked, at block 4308 the processor transmits backto the client device a block message. The block message can indicate tothe user of the client device that decoding of the CHIF file is notallowed, and, in some embodiments, can state the reason for blockingdecoding of the CHIF file. The process then ends at block 4318. If atdecision block 4306 the processor determines that decoding of the CHIFfile should not be blocked, the process moves to block 4310. At block4310, the processor transmits a decryption key and a decoder script tothe client device. In some embodiments, the decoder script can remotelyaccess the CHIF file on the client device to decode the CHIF filewithout transmitting the decoder script to the client device. At block4312, the client device decrypts the CHIF file using the decryption key,and decodes the CHIF file using the decoder script. At block 4314, theclient device stores the decoded CHIF content in a memory of the clientdevice. At block 4316, the client device presents the CHIF content onthe client device. The process then ends at block 4318.

FIG. 44 illustrates a CHIF file decryption and decoding system 4400 inaccordance with various embodiments of the present disclosure. As alsodescribed herein with respect to FIGS. 33, 34, 42, and 43 , clientdevices storing and using CHIF files remotely from a centralCHIF-enabled server, such as devices viewing a CHIF file in a decoder orviewer application, viewing in a web browser a webpage including anembedded CHIF file, etc., can trigger a script from the server to decodeand present the contents of the CHIF file. If a user, a CHIF file, awebsite, etc., are considered to be disallowed from decoding the CHIFfile, the decoding of the CHIF file can be blocked by the script.

The system 4400 includes a client device 4402 including a memory 4404.It will be understood that the client device 4402 can be any electronicdevice or computing device, such as that described in FIG. 65 herein.The client device 4402 can store in the memory 4404 an encrypted CHIFfile 4406. In some embodiments, the CHIF file may not be encrypted. Theencrypted CHIF file 4406 can be stored in the memory 4404, for example,when the CHIF file 4406 is created or encoded by the client device 4402,or otherwise retrieved by the client device 4402, such as from a server,from another electronic device, from a webpage loaded in a browser ofthe client device 4402, the contents of the webpage being stored in thememory 4404, or from other sources. As described with respect to thevarious embodiments herein, the encrypted CHIF file 4406 can includemetadata and a body, where the body includes data on the plurality ofcontent or one or more content types encoded into the CHIF file 4406.The metadata can also include a UUID that is unique to the CHIF file4406, and that can be used to track, and associate information with, theCHIF file 4406. In some embodiments, the UUID can be a hexadecimalnumber, such as F122 as shown in FIG. 44 .

When the client device 4402 is to decrypt and decode the encrypted CHIFfile 4406 in order to view the contents of the CHIF file 4406, in someembodiments the client device 4402 does not have access to a decoder,and instead requests decryption and decoding of the encrypted CHIF file4406 from a CHIF server 4408. The client device 4402 transmits adecryption and decoding request 4410 over a network 4412 to the CHIFserver 4408. In some embodiments, the request 4410 includes the UUID forthe encrypted CHIF file 4406, and, in some embodiments, the request 4410includes a copy of the encrypted CHIF file 4406. In some embodiments,the CHIF file 4406 can first be decrypted, and then a request includingthe UUID is sent to the CHIF server 4408. As illustrated in FIG. 44 ,the CHIF file 4406 is decoded at the server 4408 so that CHIF contentand metadata cannot be viewed by the client device 4402. The CHIF file4406 can be sent to the CHIF server 4408 in the request 4410, or theCHIF file 4406 can be retrieved by the CHIF server 4408 based on theUUID of the CHIF file 4406.

The CHIF server 4408 has stored thereon or associated therewith a script4414, such as a JavaScript or other script, for decoding and presentingcontents of CHIF files. The CHIF server 4408 also has stored thereon orassociated therewith at least one private key 4416 for decryptingencrypted CHIF files encrypted by a public key. In some embodiments, theprivate key 4416 can be a public key, or some other type of decryptionkey. It will be understood that CHIF files can be encrypted and decodedusing any combination of public or private keys, or any other form forencryption, without deviating from the scope of the present disclosure.The CHIF server 4408 uses the private key 4416 to decrypt the encryptedCHIF file 4406 into a decrypted CHIF file 4417.

The CHIF server 4408 also includes, or is communicatively connected to,a data store or database 4418 that includes a UUID registry 4420 and adomain registry 4422. In some embodiments, the UUID registry 4420 andthe domain registry 4422 can be part of the same registry. The UUIDregistry 4420 and the domain registry 4422 are used by the CHIF server4408 to determine if a CHIF file, such as the decrypted CHIF file 4417,can be decoded and presented. The UUID registry 4420 includes a list ofUUIDs that are to be blocked from decoding based on certain parameters,such as if the CHIF file is flagged for inappropriate or graphiccontent, such as violence, profanity, etc., if the CHIF file includescopyrighted material, if a license for the content contained in the CHIFfile is expired, or for other reasons. Decoding of CHIF files can alsobe blocked for other reasons, such as based on the identity of theparticular user or client device that sent the request 4410.

The domain registry 4422 is used by the CHIF server 4408 to determine ifwebsites hosting CHIF files are allowed to use the decoder script 4414.For example, the encrypted CHIF file 4406 stored in the memory 4404 ofthe client device 4402 may have been accessed on a website, and therequest 4410 includes a domain name or domain ID for the website onwhich the encrypted CHIF file 4406 is hosted. The CHIF server 4408 canthen look up the domain name or domain ID in the domain registry 4422.If the domain ID is not in the domain registry 4422, the CHIF server4408 can block decoding of the CHIF file. If the domain ID is in thedomain registry 4422, the CHIF server 4408 can review the information inthe domain registry 4422 to determine if decoding of CHIF files hostedon the domain is allowed. If the domain registry 4422 indicates thatdecoding is not allowed, such as if a license for the domain to use CHIFfiles is expired, the CHIF server 4408 can block decoding of the CHIFfile.

If decoding is determined by the CHIF server 4408 to be allowed, eitherindicated by one of or both of the UUID registry 4420 and the domainregistry 4422, the CHIF server 4408 will facilitate decoding of thedecrypted CHIF file 4417. In some embodiments, the determination as towhether to block decoding of a CHIF file may also be performed beforedecryption of the CHIF file 4406, and both decryption and decoding maybe blocked based on the determination. If decoding is determined to beallowed, the CHIF server 4408 uses the script 4414 to decode thedecrypted CHIF file 4417 into its separated content components 4428. TheCHIF server 4408 also extracts metadata 4425, such as using a metadataextractor, from the decrypted CHIF file 4417. The metadata 4425 cancontain valuable information on the decrypted CHIF file 4417, such ascontent tags, location data, sensory data, etc. This information, and,in some embodiments, other information such as the UUID and storage orretrieval location of the decrypted CHIF file 4717, can then be storedin a CHIF index 4427. The CHIF index 4427 can be continuously updated toprovide for third-parties to send search requests for CHIF files to theCHIF server 4408. In some embodiments, the CHIF server 4408 storescopies of CHIF files received, such as the CHIF file 4406 received inthe request 4410. The CHIF server 4408 can thus act as a central gatewayfor users to locate and access CHIF files.

The CHIF server 4408 then streams the decoded content 4428 in atransmission 4424 over the network 4412 to the client device 4402. Theclient device 4402 can then view, playback, or otherwise present, basedon the content types of the separate content components 4428, thecontent components 4428. The client device 4402 is thus prevented fromever storing the decrypted CHIF file, or its metadata, as the clientdevice 4402 only receives the decoded content 4428, and not themetadata. This prevents valuable metadata information from beingdistributed across multiple devices, and instead keeps the metadatastored on the CHIF server 4408. In some embodiments, the CHIF server4408 can detect an attempted decoding of a CHIF file by a client device,and determine if the attempted decoding is the work of an unauthorizedperson such as a hacker attempting to access secure content. In someembodiments, the CHIF server 4408 can detect the location or IP addressof the client device 4402, and alert authorities to the attemptinghacking. This is especially important when the CHIF files attempted tobe accessed are high-security CHIF files, such as CHIF files containinggovernmental or military data.

FIG. 45 illustrates a CHIF file decryption and decoding process 4500 inaccordance with various embodiments of the present disclosure. Theprocess 4500 can be used with any of the systems and other processesdescribed herein, such as with system 4400. The process 4500 can beperformed and executed by a processor such as the processor 6502described herein. The process 4500 begins at block 4502. At block 4502,a request is received, such as by a processor of the CHIF server 4408,to decrypt and decode an encrypted CHIF file stored by the requester,such as the client device 4402, described with respect to FIG. 44 . Atblock 4504, the processor decrypts the CHIF file using a decryption key.At decision block 4506, the processor determines if one or more CHIFregistries are available, such as the UUID registry 4420 and/or thedomain registry 4422. If the registries are not available, the process4500 loops back to decision block 4506 until the one or more registriesbecome available. In some embodiments, the process 4500 can continue oneven if the registries are not available. If the registries areavailable, the process 4500 moves to decision block 4508.

At decision block 4508, the processor determines if decoding of the CHIFfile is to be blocked, based on information in the registries. Forexample, registries such as the UUID registry 4420 and the domainregistry 4422 are used to determine if a CHIF file can be decoded andpresented. The UUID registry 4420 includes a list of UUIDs that are tobe blocked from decoding based on certain parameters, such as if theCHIF file is flagged for inappropriate or graphic content, such asviolence, profanity, etc., if the CHIF file includes copyrightedmaterial, if a license for the content contained in the CHIF file isexpired, or for other reasons. Decoding of CHIF files can also beblocked for other reasons, such as based on the identity of theparticular user or client device that sent the request in block 4502.

The domain registry 4422 is used to determine if websites hosting CHIFfiles are allowed to use the decoder script 4414. For example, the CHIFfile for which decryption and decoding is requested may have beenaccessed on a website, and the request received in block 4502 caninclude a domain name or domain ID for the website on which theencrypted CHIF file is hosted. The domain name or domain ID can thus belooked up in the domain registry 4422. If the domain ID is not in thedomain registry 4422, the processor can block decoding of the CHIF file.If the domain ID is in the domain registry 4422, the processor canreview the information in the domain registry 4422 to determine ifdecoding of CHIF files hosted on the domain is allowed. If the domainregistry 4422 indicates that decoding is not allowed, such as if alicense for the domain to use CHIF files is expired, the processor canblock decoding of the CHIF file.

If at decision block 4508 the processor determines that decoding of theCHIF file is to be blocked, at block 4510 the processor transmits backto the client device a block message. The block message can indicate tothe user of the client device that decoding of the CHIF file is notallowed, and, in some embodiments, can state the reason for blockingdecoding of the CHIF file. The process then ends at block 4518. If atdecision block 4508 the processor determines that decoding of the CHIFfile should not be blocked, the process moves to block 4512.

At block 4512, the processor decodes the CHIF file and stores thecontents of the CHIF file in a storage location. At block 4514, theprocessor extracts the metadata from the CHIF file and updates a CHIFindex, such as CHIF index 4427. The metadata can contain valuableinformation on the CHIF file, such as content tags, location data,sensory data, etc. This information, and, in some embodiments, otherinformation such as the UUID and storage or retrieval location of theCHIF file, can then be stored in the CHIF index. The CHIF index can becontinuously updated to provide for third-parties to send searchrequests for CHIF files to a CHIF server. In some embodiments, the CHIFserver stores copies of CHIF files received, such as the CHIF filereceived at block 4502. The CHIF server can thus act as a centralgateway for users to locate and access CHIF files.

At block 4516, the processor streams the content decoded from the CHIFfile in a to the client device that sent the request at block 4502. Theclient device can then view, playback, or otherwise present, based onthe content types of the separate content components, the now separatedcontent components of the original CHIF file. The process 4500 preventsthe client device from storing the decrypted CHIF file, or its metadata,as the client device only receives the decoded content, and not themetadata. This prevents valuable metadata information from beingdistributed across multiple devices, and instead keeps the metadatastored on the CHIF server. The process then ends at block 4518.

FIG. 46 illustrates a CHIF file continuous enrichment system 4600 inaccordance with various embodiments of the present disclosure. Trainingof artificial intelligence (AI) models or engines can involveunsupervised or supervised learning. Unsupervised learning can includethe raw input of data into an artificial intelligence model, determiningerrors and error rates from the outputs, and updating and retraining themodel until an acceptable error rate is achieved. Supervised learningcan involve providing training data to the artificial intelligence modelthat has been previously tagged or labeled, such as by humans, which canprovide the model to be trained at least slightly better startinginputs, as the input already include certain identifying information.However, manual labeling can also introduce biases, or leave outcertain, more specific or detailed, information.

The system 4600 includes provides for enrichment of CHIF files by usingspecialized artificial intelligence engines to provide contextual datawith respect to the content of the CHIF file. The enriched CHIF file canthen be used in to provide more accurate results in a variety ofprocesses. For example, as illustrated in FIG. 46 , the system 4600includes a CHIF file 4602. The CHIF file 4602 includes metadata 4604,and content 4606 including an image 4608, audio 4610, text 4612, andother information 4614, such as video or other data types. The metadata4604 can include various information such as a UUID for the CHIF file4602 and can delineate how the CHIF file is constructed for laterdecoding, as described in the various embodiments herein.

The system 4600 provides for a means to further enrich the metadata withcontextual information for the content 4606 of the CHIF file 4602. Thecontent 4606 of the CHIF file 4602 can be decoded or extracted from theCHIF file 4602 and processed through one or more specialized artificialintelligence models 4616, to tag and/or label, or otherwise provideinformation concerning, the content 4606. For example, the audio 4610can be processed by a various artificial intelligence models 4616trained for detecting various properties of audio files, such as mood(emotion in voice or mood of a song, etc.), transcription of audio intotext, song detail detection (artist, song name, etc.), or other data canbe derived from the audio 4610 by the one or more of specializedartificial intelligence models 4616. In some embodiments, transcriptionsof audio can be stored as extra text content in the content 4606 of theCHIF file 4602. Other content types such as the text 4612 and the otherinformation 4614 can be processed in this way as well. For instance, thetext 4612 in the CHIF file 4602 can be processed by the specialized AImodels 4616 trained to extract keywords from text while removing other,non-critical words (article adjectives, proposition, etc.), the text4612 can be translated by the specialized AI models 4616 into anotherlanguage, etc.

Sensory data 4618 can also be detected and added to the metadata 4604.For example, a camera system in a warehouse that captures images ofitems in the warehouse can also be configured to capture audio that issent through the specialized AI 4616 to provide extra information on theaudio. The camera system can also capture sensory data such astemperature, such as if items in the warehouse are to be stored within acertain temperature range. The temperature can then be added to themetadata 4604. In this way, the CHIF file 4602 is enriched to containeven more context data for the content. The images of the warehousewould now be associated with audio in the warehouse (with tags from theAI models 4616), a temperature of the warehouse, and other sensory datato provide for a fully-contained snapshot of the warehouse environmentat a point in time.

This provides an improvement over storing this data separately, even ifrelated, such as in a relational database, because, even though the datamay be associated, an association or certain data may change that taintsthe data. For example, in a law enforcement data system, a retina scanmay become incorrectly associated with an individual that does notactually possess the matching retina, leading to a false identification.If the data is instead stored in a self-contained CHIF file pertainingto an individual, data associations between storage locations do nothave to be relied upon, since all data on the individual can be storedin a single CHIF file. Other sensory information that can be gatheredinclude scent detection, radiation, geographical location, altitude,vibrations, water level, or any other type of sensory information thatcan be detected by sensor or other devices.

In some embodiments, the CHIF file can be retained as an immutable file,wherein the CHIF file is not allowed to be modified. In suchembodiments, to add the enriched data to the CHIF file, the CHIF filecan be decoded and the original CHIF file deleted, the metadata updatedwith the enriched content, and a new CHIF file is created from thedecoded content and including the enriched data. In some embodiments,the CHIF file can be a mutable file that can be modified to add enricheddata to the CHIF file. In such embodiments, the CHIF file may only beeditable by applications that have been granted permission to edit theCHIF file for security reasons. In some embodiments, a designated areaof the CHIF file can be editable to insert enriched data into thedesignated area of the CHIF file. In some embodiments, the CHIF file canbe added to a binder file, as disclosed herein, that includes the CHIFfile and other data or files, such as other CHIF files, that include theenriched data. In embodiments that use a binder file, the CHIF file canthus be unaltered while providing enriched data in the binder file that,by virtue of being included in the binder file, is associated with theCHIF file.

This wealth of extra data from the specialized AIs 4616 and the sensoryinformation 4618 is added to the metadata 4604 of the CHIF file 4602,such as in the form of tags (mood: somber, song name: Elegy, text tag:speech; funeral, temperature: 35° F., etc.), to enrich the CHIF file4602 with contextual information describing the content. The enrichedCHIF file 4602 can then be provided to various processes such as anobject AI model 4620, an analytics engine 4622, and/or an indexingprocess 4624. This can lead to improved quantity of data on the CHIFfile 4602, as well as improved quality of CHIF file data.

For example, if the image 4608 in the CHIF file 4602 is an image of afuneral on a beach in the winter, an object recognition AI 4620 may onlydetect that the image includes a beach with people on the beach, withoutfurther context indicating that the image includes a funeral or that itis during the winter. With the enriched data provided in the metadata4604 of the CHIF file 4602, tags including that the audio mood issomber, that the text includes keywords related to funerals, and thatthe temperature is 35° F. (and possible coupled with a geographicallocation), the object AI model 4620 can further contextualize andcategorize the image 4608 as being not just an image of a beach withpeople, but an image of a funeral on a beach in the winter. The enrichedCHIF files 4602 therefore provides for a higher quality training samplefor the object AI model 4620, and the object AI model 4620 can thus betrained to accept enrich files as input for more accurate and highlycontextualized results.

The enriched CHIF file 4602 also provides extra data for the analyticsengine 4622 to provide more accurate and informed analytics on thecontent of the CHIF file 4602, and provides more detailed indexing andsearching of the CHIF file to the indexing process 4624. In someembodiments, the indexing process 4624 can provide for searchable CHIFfiles, such as with respect to the indexes describes in FIGS. 36, 37,and 44 of the present disclosure. As additional content is added to theCHIF file 4602, the enrichment shown in FIG. 46 can further provideinformation to the metadata 4604 on that additional content. Thus, theCHIF file 4602 can be continuously enriched. Metadata of the CHIF fileformat disclosed herein thus provides a flexible data structure that canbe continuously enriched and updated to increase data quantity andquality of the CHIF file, and enhance the accuracy of processes andsystems that utilize the CHIF file.

FIG. 47 illustrates a CHIF file continuous enrichment process 4700 inaccordance with various embodiments of the present disclosure. Theprocess 4700 can be used with any of the systems and other processesdescribed herein, such as the system 4600. The process 4700 can beperformed and executed by a processor such as the processor 6502described herein. The process 4700 begins at block 4702. At block 4702,the processor receives a CHIF file including a plurality of data of aplurality of data types. At block 4704, the processor extracts one ofthe plurality of data of one of the plurality of data types. At block4706, the processor provides the extracted data to at least onespecialized artificial intelligence model. At block 4708, the processorreceives one or more outputs from the specialized artificialintelligence model and stores at least a portion of the output in themetadata of the CHIF file.

At decision block 4710, the processor determines if sensory data isavailable. If not, the process 4700 moves to decision block 4714. If so,the process 4700 moves to block 4712. At block 4712, the processorstored at least a portion of the sensory data in the metadata of theCHIF file, and the process 4700 moves to decision block 4714. Atdecision block 4714, the processor determines if there is additionalCHIF file data or content to process to provide further enrichedmetadata. If so, the process 4700 loops back to block 4704 to extractadditional data from the CHIF file to enrich. If at decision block 4714the processor determines that no additional CHIF file data is to beprocessed, the process 4700 moves to block 4716.

At block 4716, the processor provides the now enriched CHIF file to animage recognition AI model to use an image in the CHIF file for trainingor for runtime categorization and labeling, as also described withrespect to FIG. 46 . At block 4718, the processor creates or updates anindex using the enriched CHIF file, as also described with respect toFIG. 46 . At block 4720, the processor performs data analytics using theenriched CHIF file, as also described with respect to FIG. 46 . Theprocess then ends at block 4722.

FIG. 48 illustrates a CHIF previewing architecture 4800 in accordancewith various embodiments of the present disclosure. The architecture4800 includes a CHIF database engine 4802. The CHIF database engine 4802can run locally on a client device, or remotely on a server, such as theCHIF server disclosed in various embodiments herein. In someembodiments, the CHIF database engine can be an obfuscated script suchas a Javascript that handles decoding and streaming of files. The CHIFdatabase engine can include a decrypter/decoder 4804 and an API 4806.The API 4806 can be a script function such as a Javascript function. Thedecrypter/decoder 4804 can access a CHIF container 4808 to be decoded.The CHIF container 4808 can include various encoded components includingat least one header, metadata, an image, audio, text, or othercomponents as disclosed in the various embodiments herein. Thedecrypter/decoder 4804 can also include a decryption enforcement script4810 that can review requests for access or decoding of the CHIFcontainer 4808 and either grant or deny such requests, as disclosedherein, for example, with respect to FIGS. 42-45 .

A CHIF viewer 4812, such as an application installed on a client deviceor a web browser with CHIF viewing functionality, can present or displaydecoded content from CHIF files. Prior to decrypting and decoding of theCHIF container 4808, the CHIF viewer 4812 can display a preview 4814 ofthe contents of the CHIF container 4808, streamed via a streamPreviewAPI function 4816 from the CHIF database engine 4802 to the CHIF viewer4812. The streamPreview API function 4816 pulls data from the headerand/or metadata of the CHIF container 4808, and provides or transmitsthis data to the CHIF viewer 4812 to present the preview 4814. Thepreview 4814 can include, for example, preview images, text, audio, orother data stored in the CHIF container 4808.

For example, if the CHIF container 4808 includes an image and audiodata, wherein the audio data is to be played simultaneously with thedisplay of the image, the preview 4814 can include only the image. Theuser then would interact with the preview image to initiate decoding ofthe CHIF container 4808 to provide the full contents of the CHIFcontainer 4808 for presentation in the CHIF viewer 4812. For example, auser could touch or click on the preview image, prompting decoding andplayback of the audio data with the image. The preview 4814 can alsoinclude alternate text 4818 that provides prompts or information to theuser when the user performs an interaction with the preview 4814. Forexample, when a user places a mouse cursor over the preview 4814, thealternate text 4818 can appear with a message, such as “click to streamCHIF” or “click to find out more.” In this example, in response to aclick on the preview 4814, the CHIF database engine 4802 would decryptand/or decode the CHIF container 4808 and provide the decoded content tothe CHIF viewer 4812. To provide the decoded content to the CHIF viewer4812, a streamFiles API function 4820 can retrieve and send the decodedcontent to the CHIF viewer 4812. In some embodiments, a getPart APIfunction 4822 can retrieve certain parts or portions of the CHIF file.The decoded content provided to the CHIF viewer 4812 can include any orall of the data previously encoded into the CHIF container 4808, such asmetadata 4824, image(s) 4826, audio data 4828, text data 4830, and/orother data.

FIG. 49 illustrates an example browser window 4900 in accordance withvarious embodiments of the present disclosure. As described herein withrespect to FIG. 48 , a preview of a CHIF file can be presented beforedecoding and/or presentation of CHIF file contents. The browser window4900 includes a webpage 4902 that includes a preview image 4904. Thepreview image can be configured to display alternative text 4906 when amouse cursor 4908 moves over the preview image 4904. The alternativetext 4906 can provide additional information concerning the contents ofa CHIF file, a user action prompt, or other information. In the exampleof FIG. 49 , the alternative text 4906 prompts the user to click thepreview image 4904, which initiates decoding and displaying of CHIF filecontents. A server may decode a stored CHIF file and then transmit thecontent for presentation on the webpage 4902. The webpage 4902 caninclude other content such as user interaction buttons 4912 and textualproduct information 4910.

Presenting audio with an image in this way offers a more efficient meansof providing audio information with an image. Typically, if one wishesto provide image content in association with audio content, one wouldcreate a video file, such as an MP4 file, and lay an audio track overthe image. This may be an inefficient method of associating audiocontent with an image because if the goal is to provide audio contentwith one or more still images, rather than with moving video content,creating a video file to achieve such creates a bigger file than needed,as video files are commonly much larger than an image or an audio file,even when the image or audio file sizes are combined. The CHIF file canbe the same or a similar size to the combined size of the image andaudio files, and thus provides a more efficient file type that takes upless storage, is transmitted faster, etc. It will be understood thatthis may be the case for other file types as well, such as if a textdocument was also included in the CHIF file, the size of the CHIF filewould only increase in an amount close to that of the size of the textdocument. Additionally, the preview of the CHIF contents enables a userto quickly view the preview before deciding whether to view all of theCHIF file contents. It will be understood that the example browserwindow 4900 is merely an example of where CHIF file content can beencountered by users. CHIF file contents and associated previews canalso be presented in other applications, such as retailer applications,social media applications, app store applications, or others.

FIG. 50 illustrates a CHIF file scarcity creation system 5000 inaccordance with various embodiments of the present disclosure. Thesystem 5000 can be used to create a limited number of digital assets.The system 5000 includes at least one CHIF server 5002, at least onecreator device 5004, and at least one viewer device 5006. In variousembodiments of the present disclosure, a content creator may desire tocreate a limited quantity of copies of a CHIF file for dissemination,such as a limited music single release, a special edition book or comicbook, a limited video, or other content that can be limited by thenumber of copies. For example, a music artist may wish to release only1000 copies of a promotional single to increase demand and mystique forthe product. A digital CHIF marketplace can be managed by the CHIFserver 5002 to provide access to, and a user interface for, a repositoryof CHIF files. The creator device 5004 can access the CHIF marketplace5008 using a CHIF marketplace application 5010. The creator device 5004can use a CHIF encoder 5012 to create a CHIF file, and then upload theCHIF file to the CHIF server 5002 via the CHIF marketplace application5010. In some embodiments, the CHIF encoder 5012 can provide options forlimiting the number of copies of the CHIF file. In some embodiments,limitations on copies can be provided during the upload process of CHIFfiles using the CHIF marketplace application 5010. The creator device5004 can also include a CHIF library 5014 that can manage storage ofCHIF files locally on the creator device 5004 and/or keep track of CHIFfiles associated with a user account to display to a user of the creatordevice 5004 the CHIF files to which the user has access via the CHIFmarketplace 5008. The creator device 5004 can also include a contactslist 5016 that can be used, such as via the CHIF marketplace application5010, to share CHIF files from the creator device 5004 to the viewerdevice 5006.

When the creator device 5004 creates a CHIF file using the CHIF encoder5012 and requests to upload the CHIF file to the CHIF marketplace 5008via the CHIF marketplace application 5010, the creator device 5004transmits over a network 5018 the created CHIF file to the CHIFmarketplace 5008 provided by the server 5002. The CHIF file is thensaved in CHIF storage 5020. The CHIF file can also be assigned a UUIDeither at the time of creation by the CHIF encoder 5012, or upon uploadto the server 5002. The UUID of the CHIF file can be associated with thecreator of the CHIF file, or with other data, in a registry 5022 storedin a database 5024. If the CHIF file is to be a limited copy CHIF file,a series of UUIDs, such as a series of sequential UUIDs, can beallocated or reserved for use by CHIF file copies and stored in theregistry 5022. For example, as shown in FIG. 50 , the registry 5022 inthis example includes 1000 reserved UUIDs F1-F1000 associated withcontent titled “Limited Edition X.” For each UUID in the registry 5022,a status such as “issued” or “inactive” can be associated with the UUID,wherein “issued” designates that the UUID is already in use, and thus acopy has been created, while “inactive” designates that a copy has notyet been created and a UUID not yet assigned. It will be understood thatthe status can be listed in other ways, such as using an assigned flagwith a binary 0 (unassigned) and 1 (assigned).

The registry 5022 can also store in association with UUIDs a sequencenumber that can either be a single number, or a number out of a totalnumber as shown in FIG. 50 . For example, UUID F1 in FIG. 50 shows asequence number of 1/1000, indicating F1 is the first copy out of 1000possible copies. When the max number of copies have been reached, theserver 5002 can deny further requests via the marketplace for copies. Insome embodiments, instead of reserving a series of sequential UUIDs, aseparate sequential number can be assigned to a CHIF file when createdbased on examining the registry 5022 to determine how many CHIF filesfor a given limited content have previously been created. In such anembodiment, the UUID may not be sequential, but the registry 5022 andmetadata of the CHIF file can include the sequence number for the CHIFfile. The UUID and the sequence number, if any, can both be stored inthe registry and in CHIF file metadata. The registry 5022 can alsoassociate CHIF files with a user account such as a unique account nameor account identifier. Associating the CHIF file to an account ensuresthat only users granted access to the copy of the CHIF file can view theCHIF file when logged in. That is, if a CHIF file is copied to anotherdevice that is not logged in to the associated user account, attempteddecoding and viewing of the CHIF file can be blocked.

After a limited copy CHIF file is created and stored on the server 5002,the viewer device 5006 can browse the CHIF marketplace 5008 via a CHIFmarketplace application 5010 installed on the viewer device 5006, oraccessed via a web browser. The viewer device 5006 can then select aCHIF file for download and/or purchase, and send a request for theselected CHIF file to the CHIF server 5002. Upon the CHIF marketplaceservices 5008 on the CHIF server 5002 receiving the purchase requestfrom the viewer device 5006, the server 5002 determines a storagelocation in the CHIF storage 5020 for the requested CHIF file, andconsults the registry 5022 to find the next available UUID and/orsequence number to be used for the requested CHIF file. The server 5002then alters the registry to mark the available UUID as issued. In someembodiments, the server 5002 can then transmit the CHIF file to theviewer device 5006 for download by the viewer device 5006, which candecode and view the CHIF file in a CHIF viewer 5026. In someembodiments, for example as shown in FIG. 50 , the server 5002 decodesthe CHIF file via a CHIF decoder 5028 executed by the server 5002, andtransmits the decoded content 5030 to the viewer device 5006 for viewingvia the CHIF viewer 5026. In some embodiments, the server 5002 can alsogenerate a number of copies according to the copy limit for storage inthe CHIF storage 5020, each having a UUID and sequence number in theCHIF file metadata.

The viewer device 5006 can also include a CHIF library 5032 that canmanage storage of CHIF files locally on the viewer device 5006 and/orkeep track of CHIF files associated with a user account to display to auser of the viewer device 5006 the CHIF files to which the user hasaccess via the CHIF marketplace 5008. The viewer device 5006 can alsoinclude a contacts list 5034 that can be used, such as via the CHIFmarketplace application 5010, to share CHIF files from the creatordevice 5004 to the viewer device 5006. For example, in some embodiments,a copy of a limited copy CHIF file can be shared from one device toanother, such as between the creator device 5004 and the viewer device5006. For example, the user of the viewer device 5006 can be in thecontact list 5016 of the creator device 5004, and the creator device5004 can share a copy of the CHIF file to the viewer device 5006 via anapplication with access to the contacts list 5016, such as the CHIFmarketplace application 5010. As another example, an artist may wish toshare free samples of a limited promotional single, and send an email tousers in a fan list with a link to download the single via themarketplace. Other methods of sharing the CHIF file can include viaemail, short message service (SMS) or text message, or chatapplications. When the creator device 5004 shares the CHIF file with theviewer device 5006, in some embodiments the creator device 5004 cancreate a CHIF file with a new UUID and/or sequence number, and thennotify the server 5002 of the sharing event so that the server 5002 canupdate the registry 5022 accordingly using the UUID and/or sequencenumber in association with the particular item of content. The creatordevice 5004 can then transmit the CHIF file to the viewer device 5006.In some embodiments, instead of transmitting the CHIF file from thecreator device 5004 to the viewer device 5006, the creator device 5004can create and upload to, or request an already uploaded CHIF file from,the server 5002, and the server 5002 provides the shared CHIF file tothe viewer device 5006, such as via the CHIF marketplace 5010. Thesharing notification from the creator device 5004 also prompts theserver 5002 to check the registry 5022 to determine if the CHIF file hasreached its copy limit, and, if so, can deny the share event.

FIG. 51 illustrates a CHIF file scarcity creation process 5100 inaccordance with various embodiments of the present disclosure. Theprocess 5100 can be used with any of the systems and other processesdescribed herein, such as the system 5000. The process 5100 can beperformed and executed by a processor such as the processor 6502described herein, and/or by a server such as the CHIF server 5002. Theprocess 5100 begins at block 5102. At block 5102, the processor receivesuploaded content, such as a newly created CHIF file, that includes aspecified copy limit. At block 5104, the processor generates a set ofunique identifiers and/or sequence numbers according to the specifiedcopy limit, such as that described with respect to FIG. 50 . Theprocessor updates a CHIF file registry with the set of uniqueidentifiers and/or sequence numbers associated with the content includedin the CHIF file, to track the creation, copying, and sharing of theCHIF file, such as described in FIG. 50 . At block 5106, the processorprovides the CHIF file for download in a CHIF marketplace and tracks thedownloading of CHIF files from the marketplace, and otherwise tracks thesharing of CHIF files, such as sharing between client devices.

At block 5108, the processor receives a request via the marketplace, orreceives a sharing notification, indicating a request or sharing eventfor a copy of the uploaded CHIF file. At decision block 5110, theprocessor determines if the copy limit has been reached, that is, thenumber of downloads/purchases and shares of the limited CHIF fileexceeds the specified copy limit. If so, at block 5112, the processorissues a denial notification that is transmitted to the device thatrequested the CHIF file at block 5108. The process then ends at block5126. If, at decision block 5110, the processor determines that the copylimit has not been reached, the process 5100 moves to block 5114. Atblock 5114, the processor records a new copy of the CHIF file as issuedand as being assigned or associated with the requesting user account,all in association with an assigned sequential number and/or uniqueidentifier of the CHIF file, which information can be recorded in theregistry accessible to the server. At block 5116, the processorfinalizes the purchase or sharing, and transmits, or allows transmissionof, the CHIF file to the requesting device.

At block 5118, the processor, such as via a decoder script that notifiesthe processor of decoding and/or viewing attempts for CHIF files,detects an attempted viewing of the CHIF file. At decision block 5120,the processor determines if there is a user account associated with theattempted viewing, and, if so, if the user account is allowed access tothe CHIF file based on the data associated with the UUID of the CHIFfile previously recorded. If the user account is not allowed access, atblock 5122, the processor issues a denial notification. The process 5100then ends at block 5126. If, at decision block 5120, the processordetermines that the user account is allowed access to the CHIF file, theprocessor, at block 5124, decodes, or allows another device, such as therequesting device, to decode, the CHIF file for viewing of the contentsof the CHIF file. The process 5100 then ends at block 5126.

FIG. 52 illustrates an example data mobility and edge computing system5200 in accordance with various embodiments of the present disclosure.Existing systems for providing information and/or offers to users viamobile devices are limited in that the existing systems often use arealtime database to store information and offers pertaining to usersfrom various businesses, wherein the realtime database must includeinformation and offers from all partners for all customers. This isexpensive and often risky to build and maintain. It also presents asecurity risk, as customer may not want their data aggregated and sharedwithin the realtime offer database with third-party businesses.

In addition to the disadvantages of maintaining a realtime database forthird-party information and offers, traditional cloud computing in whichcontent storage and online computations are performed by remote serversthat send and receive data from client devices can have poor performancedue to latency. The time it takes for data to travel over even fibernetworks connecting servers to radios in towers or base stations, clientdevices, or other network devices often provides for poor userexperience, as the user must wait for communications to travel therequired distance to and from the servers. To counter poor performanceof traditional cloud computing, and with the advent of 5G networks, thevarious embodiments of this disclosure can incorporate edge computingstrategies to decrease latency and improve the user experience for userscreating, receiving, and viewing CHIF files. Edge computing, ormulti-access edge computing (MEC), as provided in the variousembodiments herein, incorporates a network architecture that enablescloud computing to be performed at the edge of a network, such as near abase station, at CDN edge cache servers, or on client mobile devices.Online computations can therefore be performed closer to end users,speeding up communications between the users and service providers.

While edge computing can provide the above benefits, there exist issueswith edge computing solutions. The distributed nature of edge computingintroduces a shift in security schemes used in cloud computing.Different encryption mechanisms should be implemented for edge computingsystems, since data may transit between different distributed nodesconnected through the Internet before eventually reaching the cloud.Edge nodes or devices may also be resource constrained devices, limitingthe choice in terms of security methods. Additionally, edge computingresults in a shift from a centralized top-down infrastructure to adecentralized trust model. An advantage of edge computing, however, isthat data can be kept at the edge, shifting ownership of collected datafrom service providers to end-users. The various embodiments of thepresent disclosure can address the security issues of edge computing byproviding data to edge devices in secure, encrypted, and self-containedCHIF files. The various embodiments of the present disclosure also allowat least portions of data to be retained and cached by edge devices,such as CDN edge cache servers, 5G base stations, and user mobiledevices.

The data mobility and edge computing system 5200 includes one or moreCHIF servers 5202 that include one or more CHIF storages 5204 forstoring CHIF files to be provided to edge devices and/or end users.While the CHIF servers 5202 can provide various services that involvethe creation and transmission of CHIF files for use by edge devices andclient devices, partner service providers 5206 can also offer variousservices through the CHIF servers 5202 such as travel arrangementservices, advertising services, content management services such asmusic library store, management, and streaming services, special retailoffers for subscribing users, or any other services that can be offeredby third-parties to end users. The CHIF servers 5202 can act as a middleman for these offered services, receiving and storing CHIF filescontaining offers, advertisements, purchased content, or other services.The CHIF servers 5202 can also provide an integrated user marketing andpreferences platform. Consent can be provided in advance from users forvarious services provided by the partner service providers 5206, so thatusers only receive CHIF files, or decoded CHIF file content, fromauthorized partner service providers 5206.

Additionally, the CHIF server 5202, or an edge device 5208 in someembodiments, can detect events that trigger a request from the CHIFservers 5202 for offers, information, or other services from the partnerservice providers 5206. For example, if a user purchases a physical ordigital good from a retailer, the edge device 5208 and/or the CHIFserver 5202 can be notified, and in turn request offers from relevantpartner service providers 5206, such as a partner service provider 5206associated with the retailer from which the physical or digital good waspurchased, or partner service providers 5206 that offer goods orservices related to the purchased physical or digital good. As anotherexample, if an end user misses a travel arrangement, such as a flight,the user's mobile device 5210 can send a notification to an edge device5208 or to the CHIF server 5202. The edge device 5208 and/or the server5202 can then request travel arrangement offers from the partner serviceproviders 5206, such as alternative flight options, hotel bookings,restaurant reservations, or other offers.

To reduce latency and facilitate quicker response times for users tosend notifications regarding events, receive information, offers, etc.,from partner service providers 5206, or send responses to theinformation, offers, etc., CHIF files containing the information,offers, etc. can be provided to the edge device 5208 from the CHIFserver 5202. The edge device 5208 can also be provided with thenecessary assets and code to decode CHIF files, present the information,offers, etc. to the mobile device 5210, such as via a web browser orother type of application on the mobile device, and receive responsesfrom the mobile device 5210 regarding the information, offers, etc. Theedge device 5208 can, in some embodiments, receive CHIF files directlyfrom partner service providers 5206. In various embodiments disclosedherein, the edge device 5208 can be one or more CDN edge cache serversthat reside at various locations for providing service to nearby endusers, base stations such as 5G radio towers, or other edge computingdevices. Therefore, instead of the mobile device 5210 being required totransmit and receive all responses regarding the information, offers,etc., to and from a remote server such as the CHIF server 5202, themobile device 5210 can communication with the edge device 5208. The edgedevice 5208 can thus store or cache the assets and code necessary todecode data from the CHIF files, provide the information, offers, etc.from the CHIF files to the mobile device 5210, and receive messages fromthe mobile device 5210 regarding the decoded data, such as selecting oneof the offers. The selection can then be transmitted to the partnerservice provider 5206 associated with the selected offer to facilitateservice to the mobile device 5210 for the selected offer.

For example, if a CHIF file provided by a partner service provider 5206including an offer for alternative travel accommodations is sent to theedge device 5208, the edge device can use cached assets and code todecrypt and/or decode the CHIF file, and transmit the decoded CHIF filecontents to the mobile device 5210 for viewing by the mobile device5210. In various embodiments disclosed herein, the CHIF server 5202 candetect a location of the mobile device 5210, and route CHIF filesreceived from partner service providers 5206 to one or more edge devices5208 near the mobile device 5210, so that the mobile device 5210receives service with reduced latency and to provide the user with anenhanced user experience.

In some embodiments, the mobile device 5210 can act as an edge computingdevice. CHIF files can be transmitted from the CHIF server 5202, or fromone or more partner service providers 5206, to the mobile device 5210,along with the assets and code necessary to decrypt and decode the CHIFfiles. The mobile device 5210 can then decrypt and/or decode CHIF filesto view the information, offers, etc., and respond to the information,offers, etc., either to the CHIF server 5202, or directly to the partnerservice providers 5206. In some embodiments, the assets and codenecessary to decrypt and decode CHIF files can be stored or cached onthe mobile device 5210. Therefore, the mobile device 5210 need onlyreceive one communication including a CHIF file, and can then view theCHIF file contents and send one response back, if any, rather thansending and receiving multiple communications between the mobile device5210 and a remote server, such as the CHIF server 5202. In someembodiments wherein CHIF files provided by the partnered serviceproviders 5206 are immutable, updated CHIF files, such as if an offerexpires, including new information can be transmitted by the partneredservice providers 5206. In some embodiments wherein CHIF files aremutable containers, the original CHIF file can be edited with the offerinformation.

FIG. 53 illustrates an example data mobility and edge computing process5300 in accordance with various embodiments of the present disclosure.The process 5300 can be used with any of the systems and other processesdescribed herein, such as the system 5200. The process 5300 can beperformed and executed by a processor such as the processor 6502described herein, and/or by a server such as the CHIF server 5202, or aprocessor of an edge device such as edge device 5208. The process 5300begins at block 5302. At block 5302, the processor detects an event thattriggers sending of CHIF files to a user, such as a new purchase by theuser, a user checking in at a location, a travel delay or change, orother events. At block 5304, the processor receives a plurality of CHIFfiles including various information, service options, offers,advertisements, or other content from a plurality of partnered serviceproviders, related to the event detected in block 5302. At block 5306,the processor causes the plurality of CHIF files to be stored, such asin a storage device of a CHIF server. In some embodiments, the processormay not cause the plurality of CHIF files to be permanently stored, butmay instead temporarily store the CHIF files long enough to transmit theCHIF files to an edge device or a user's mobile device.

At decision block 5308, the processor determines if a user's mobiledevice is near an edge device, such as a CDN edge server or a basestation. If not, at block 5310, the processor causes the relevant CHIFfiles previously stored to be transmitted to the user's mobile device,along with any code or assets the mobile device can use to decrypt anddecode the relevant CHIF files. In some embodiments, the mobile devicemay already have cached the assets and code used to decrypt and decodethe relevant CHIF files. The processor then moves to block 5320, wherethe user views the decoded CHIF file contents on the mobile device. Insome embodiments, if the mobile device is not to be used as an edgedevice, the processor can cause the server to decode the CHIF filecontents and transmit the CHIF file contents to the mobile device forviewing by the mobile device. If, at decision block 5308, the processordetermines that the mobile device is near an edge device, such as a CDNedge server or a base station, the process 5300 moves to block 5312. Atblock 5312, the processor causes the relevant assets, code, and CHIFfiles to be transmitted to the edge device. At decision block 5314, theprocessor and/or the edge device determines if the CHIF files should bedecoded at the edge device. If not, at block 5316, the edge devicetransmits data including the CHIF files and any items needed to decryptand decode the CHIF files locally on the mobile device, such asencryption keys and decoding scripts. The process 5300 then moves toblock 5320.

If, at decision block 5314, the processor and/or the edge devicesdetermines that the CHIF files are to be decrypted and/or decoded by theedge device, the process 5300 moves to block 5318. At block 5318, theedge device caches the assets and code, if such has not already beencached before, decrypts and/or decodes the received CHIF files, andtransmits the decoded content to the mobile device. At block 5320, themobile device views the decoded content, such as advertisements, offers,service provider information, etc. At block 5322, the processor and/orthe edge device receive a response from the mobile device based on thevarious information and service options provided to the mobile deviceuser via the transmitted CHIF files. In some embodiments, the mobiledevice can send a response directly to a partnered service provider. Theresponse can trigger other events based on the information, offers, etc.provided in the CHIF files, such as purchasing a good, booking a travelarrangement, or other events. At block 5324, after a predeterminedperiod of time has elapsed, the CHIF files transmitted to the edgedevice and/or the mobile device expire, so that out-of-date information,offers, etc. cannot be accepted or responded to by users. The process5300 ends at block 5326.

FIG. 54 illustrates an example travel arrangement offers and edgecomputing system 5400 in accordance with various embodiments of thepresent disclosure. As described with respect to FIGS. 52 and 53 , edgedevices can be utilized to provide services to mobile devices withreduced latency and thus an improved user experience. This can be evenmore important when a user needs to quickly make new travelarrangements, such as if the user misses a flight or othertransportation option. In this example, if a user misses a flight, theuser's mobile device 5410 can send a notification of the missed flightto a base station 5408 that is nearby to the travel hub, such as a basestation near an airport. The base station 5408, in some embodiments, canalready have cached thereon assets and code used for decrypting anddecoding CHIF files and/or providing other travel options to the user.In some embodiments, a CHIF storage 5402 can receive offers secured inCHIF files for alternate travel arrangements from partnered serviceproviders, such as one or more airlines 5405, one or more hotels 5406,and/or one or more other business types 5407, which can be the partneredservice providers 5206 described in FIG. 52 . The CHIF files received bythe CHIF storage 5402 can then be provided to the base station 5408. Insome embodiments, if the base station 5408 does not already have cachedthereon assets and code for manipulating CHIF files and for providingthe offers to the mobile device 5410, the CHIF storage 5402 can alsosend the assets and code to the base station 5408. The CHIF storage 5402can, in some embodiments, be the CHIF server 5202 described in FIG. 52 .In some embodiments, the base station 5408 can receive CHIF filesdirectly from the partnered service provider without using the CHIFstorage 5402. In some embodiments, the CHIF storage 5402 can beassociated with or be part of the base station 5408.

As the base station 5408 receives the CHIF files from the partneredservice providers, the base station 5408 can either provide the CHIFfiles to the mobile device 5410 for decoding of the CHIF files, such asin a mobile application or a webpage in a web browser in which the userselects offers and a script decodes the CHIF file for the offer topresent the offer information to the user, or the base station 5408 candecode the CHIF files, the contents of which are then presented to theuser. The offers presented to the user can include, in this example,flight options, hotel reservation options, restaurant reservationoptions, or other options related to travel. The code cached on the basestation can also be configured to present options, timelines, sortedinformation, and otherwise assist the customer with making a decisionbased on the available offers. At the end of the session between thebase station 5408 and the client device 5410, or at any otherpredetermined time, the CHIF files can expire, and all decoded contentsdeleted. This prevents data from being permanently moved or stored onthe base station 5408 or the mobile device 5410.

FIG. 55 illustrates an example travel arrangement offers and edgecomputing process 5500 in accordance with various embodiments of thepresent disclosure. The process 5500 can be used with any of the systemsand other processes described herein, such as the system 5400. Theprocess 5500 can be performed and executed by a processor such as theprocessor 6502 described herein, and/or by a server such as the CHIFserver 5202, or a processor of an edge device such as edge device 5408.The process 5500 begins at block 5502. At block 5502, the processordetects an event from a user's mobile device indicating the user misseda travel arrangement. In some embodiments, the detection of the eventcan be due to a notification from the user's mobile device, while, insome embodiments, the processor can track the user's travel arrangementsand can detect if a user does not board a flight, check-in to a hotel bya certain point in time, etc.

At decision block 5504, the processor determines if there are anycurrent offers available. In some embodiments, limited time offers canbe provided from partnered service providers even when there is not aknown event such as a missed travel arrangement, in which case, if theoffers have not expired, the processor can retrieve the offers for theuser of the mobile device. If there are current offers available, atblock 5506, the processor retrieves one or more current offers from aCHIF storage. The process 5500 then moves to block 5510. If, at decisionblock 5504, the processor determines that there are not currently anyrelevant offers to provide to the user, the process 5500 moves to block5508. At block 5508, the processor requests and receives one or moreoffers for travel arrangements stored securely in CHIF files frompartner service providers. At block 5510, the processor detects that theuser's mobile device is located in proximity to a base station near atravel hub. It will be understood that the base station can be anotheredge computing device, such as one or more CDN edge servers. At block5512, the processor causes the CHIF files, whether retrieved at block5506 or received at block 5508, to be transmitted to the base station.In some embodiments, assets and code for the decoding of CHIF files andfor presenting offers to the user can also be transmitted to the basestation. In some embodiments, this data may already be cached at thebase station, and only the CHIF files are sent. In some embodiments,instead of detecting a user's mobile device is near a base station, andthen transmitting the CHIF files to the base station, a base stationthat already has cached assets and code for providing travel arrangementoffers to users can detect the event in block 5502, and retrieve orrequest offers directly from service providers.

At block 5514, the base station decrypts and decodes the CHIF filescontaining the offers, and, at block 5516, the base station transmitsthe decoded offers to the mobile device for presentation of the offersto the user, such as in a mobile application or a web browserapplication. In some embodiments as described herein, the CHIF files canbe transmitted to the mobile device and decoded at the mobile device. Atblock 5518, the processor receives a response from the mobile devicerelated to one of the present offers, which can trigger further eventssuch as booking an alternative flight option, booking a hotel, making arestaurant reservation, calling a cab, or other events. At block 5520,the CHIF files in which the offers were provided expire after apredetermined period of time, such as a number of minutes or hours, orat the end of a communication session between the base station and themobile device. The process 5500 ends at block 5522.

FIG. 56 illustrates an example CHIF file 5600 with selectively encryptedcontent in accordance with various embodiments of the presentdisclosure. The CHIF file 5600 includes metadata 5604 and content 5606.Content 5606 can include image data 5608, audio data 5610, text data5612, and other data 5614, such as video data or other data types orfile types. The metadata 5604 can include various information such as aUUID for the CHIF file 5600 and can delineate how the CHIF file 5600 isconstructed for later decoding, as described in the various embodimentsherein.

As illustrated in FIG. 56 , the text data 5612 can include bothunencrypted text and encrypted text. During creation of the CHIF file5600, certain portions of the text data 5612 can be selected forencryption, such that certain text data is encrypted, and other textdata remains unencrypted and available for public inspection upondecoding of the CHIF file. Once the CHIF file 5600 is decoded, theencrypted text data can be decrypted if a user or a user's device isable to generate, retrieve, or access an associated decryption key. Insome embodiments, the text data 5612 comprises data structures thatinclude specific text. For example, a data structure included in thetext data 5612 can be a string data structure that includes a person'sfirst and/or last name. As another example, a data structure included inthe text data 5612 can be an array of text data that includes severalassociated textual data components. For instance, the array couldinclude medical testing diagnostic information including a testing type,such as a viral test type, e.g., influenza, a test result, e.g.,positive or negative, a date first contracted, a testing location,and/or other information. This array could be encrypted or remainunencrypted. For example, the data on just the testing results in thearray could remain unencrypted and so that this information in the CHIFfile can be easily indexed and searched, while other information, suchas personally identifiable information (PII) stored in other datastructures, can be encrypted. This creates a public information layerthat can be seen as long as the user can decode the CHIF file, and asecure and immutable personal information layer that requires the properencryption key to access. In some embodiments, a tag data layer 5605 canalso include tags that identify or categorize content, such as testingresults information, or other types of information, and can be mutablesuch that this tag data can be changed if needed. For example, the tagdata layer 5605 can include tags related to disease data such as adiagnosis, treatment plans, prognosis, etc. with a public classificationin an unencrypted state for open examination and aggregate analysis.Encrypted data in the CHIF file 5600 can include a patient's personallyidentifiable data with PII or HIPPA classification and encrypted with adesignated encryption key so only certain roles can access the encrypteddata to protect this more sensitive data. In some embodiments, the tagdata layer 5605 can be within one of the content areas of the CHIF file5600, such as the image data 5608, audio data 5610, the text data 5612,or other content types. In some embodiments, such as illustrated in FIG.56 , the tag data layer 5605 can be a layer separate from the content5606.

Selective encryption thus allows for different data to be encryptedbased on data type classifications, and, during creation and encoding ofa CHIF file, the specific data components of the text data 5612 can beselected for encryption and encrypted with an encryption key. In someembodiments, each data structure can have a key value and, if a datastructure is selected for encryption, the key value for the datastructure is stored in a key value store 5607 of the CHIF file 5600. Insome embodiments, the key value store 5607 also includes encryptiondata, such as an initialization vector or hash function used inconjunction with a passphrase provided by a user to generate anencryption key to encrypt the selected data to be encrypted. When theencrypted information in the CHIF file 5600 is to be decrypted, a useragain provides the passphrase, which, if correct, is used in conjunctionwith the initialization vector to decrypt the selectively encryptedcontent. In some embodiments, key stretching can also be used by alsousing a random salt, or key strengthening can be used, such as byextending the key with a random salt and then deleting the salt,requiring a legitimate user to have or obtain the salt value. In someembodiments, a different passphrase can be provided for each item ofcontent to be encrypted, to differentially protect each item of contentand/or to allow for different users having different passphrases toaccess only certain encrypted content, while not being able to accessother encrypted content for which they do not have the passphrase. Insome embodiments, a different initialization vector can also be used foreach item of content. For instance, one string including a user's lastname and another string including a user's home address can each have adifferent key value and can be encrypted using a different encryptionkey generated from using different passphrases. The content key valuefor each content data structure can then be stored in the key valuestore 5607, along with the initialization vector. In some embodiments,the content key value can be stored in the metadata 5604. Once the CHIFfile 5600 is decoded, each separately encrypted data component can bedecrypted by the decoder identifying which content is encryptedaccording to the key values stored in the key value store 5607, and thedecoder can then request the passphrase for each separately encryptedcontent structure. If the correct passphrase is provided, the contentassociated with the passphrase is decrypted using the passphrase and theinitialization vector. In some embodiments, if an incorrect passphraseis provided, the passphrase may still be used to attempt to decrypt thecontent, but the resulting decrypted content would be unrecognizable orindiscernible to the user, as the incorrect passphrase would not providethe original content. In some embodiments, other portions of the CHIFfile 5600 can also be encrypted, such as the image data 5608, the audiodata 5610, and/or the metadata 5604.

FIG. 57 illustrates an example CHIF selective encryption system 5700 inaccordance with various embodiments of the present disclosure. Thesystem 5700 includes a user system 5702. The user system 5702 can be anyuser device capable of encoding and/or decoding CHIF files. The usersystem 5702 can have stored thereon, or at least stored in a storagelocation accessible by the user system 5702, a plurality of content suchas text data, audio data, video data, image data, or other data asdescribed in the various embodiments herein. As illustrated in theexample of FIG. 57 , the user system 5702 stores a plurality of dataincluding content data 5706, 5708, and 5710. The plurality of data canbe organized to be embodied in various data structures. For example,data can be contained in various data structures such as strings,arrays, stacks, trees, linked lists, graphs, various numericalstructures such as integers, floats, etc., graphs, or other datastructures. To track content and to facilitate and organize how data isto be encrypted, the plurality of data on the user system 5702 can beassociated with content key values, such that a data structure isassociated with a content key value and can be accessed or retrieved byits content key value. For example, as illustrated in FIG. 57 , contentdata 5706 is associated with a first content key value 5712, contentdata 5708 is associated with a second content key value 5714, andcontent data 5710 is associated with a third content key value 5716.Thus, for instance, content data 5706 can be accessed, retrieved, orotherwise used by referring to the first content key value 5712. Thisassociation between content data and content key values can also beuseful in presenting information to users of the user system 5702, suchas providing information in a fielded form that loads content into theform by retrieving content for each field according to content keyvalues assigned to the fields.

In addition to allowing for content data to be presented in an organizedmanner to users, the content key values can also be used to associatecontent data with encryption data 5722 when selectively encryptingcontent for CHIF files, such as an initialization vector or hashfunction. As described in other embodiments herein, content to beencoded into a CHIF file can also be selectively encrypted. This allowsfor different data portions of the CHIF file to be protected byencryption and only accessed if a user has possession of, or access to,the appropriate decryption data, such as a passphrase. In someembodiments of the present disclosure, the selective encryption of CHIFfile content can be in a “bring your own key” (“BYOK”) arrangement, suchthat users of the user system 5702 that are creating CHIF files havetheir own methods encrypting/decrypting content, such as particular keyderivation functions. In some embodiments, a central CHIF server, suchas CHIF server 4208 or other CHIF servers disclosed herein, can storeencryption data for use by third parties that are allowed access to theCHIF server, such as via a login or authentication process.

Whether in a BYOK arrangement, a CHIF server arrangement, or in someother arrangement, the user system 5702 can use a CHIF encoder 5704stored either on the user system 5702 or on a device to which the usersystem 5702 has access. The encoder 5704 receives the content data andcontent key values, and selections for which data to encrypt. Thecontent key values 5712, 5714, 5716 and encryption data 5722 are usedwhen creating a selectively encrypted CHIF file to enable laterdecryption of CHIF file content by authorized users. For example, asillustrated in FIG. 57 , a CHIF file 5726 can be created by the encoder5704 using the content data and content key values from the user system5702, and the encryption data 5722. For instance, during creation of aCHIF file by a user of the user system 5702, the user can select fromthe plurality of content data which of the content data to encrypt, ifany, before encoding the data into the CHIF file 5726. When a userselects content data for encryption, an the encryption data 5722 is usedto encrypt the selected content data. For example, the encryption data5722 can include an initialization vector or hash function that takes apassphrase provided by a user and, using the passphrase to generate asecret key, encrypts the selected content data. In some embodiments,different passphrases can be used for certain content data, such as if afirst user has a passphrase and is meant to access content data 5708,while a second user has a different passphrase and is meant to accesscontent data 5710. Once the CHIF file 5726 is created, the first userwould only be able to decrypt and view the content data 5708 using thefirst user's passphrase, and the second user would only be able todecrypt and view the content data 5710 using the second user'spassphrase. In some embodiments, different encryption data 5722 can beused for each content data structure selected for encryption, to furtherprotect the different content data. In some embodiments, the sameencryption data 5722 can be used for more than one content datastructure, if chosen to do so by the user. When encryption data 5722 isretrieved and is used to encrypt one or more content data, the CHIFencoder 5704, as disclosed in various embodiments herein, stores theencrypted content and its associated content key value in a content dataportion 5728 of the CHIF file 5726. The CHIF encoder also stores in akey value store 5730 of the CHIF file 5726 the content key valueassociated with the stored encrypted content and the encryption data5722, such as the initialization of hash function used previously toencrypt the content. In embodiments wherein different encryption data isused for different content data, the encoder 5704 can also store in thekey value store 5730 each encryption data used in association with thecontent key value for the content that particular encryption data wasused to encrypt.

For example, as illustrated in FIG. 57 , the CHIF file 5726 encodedusing the content data from the user system 5702 includes, among otherdata, unencrypted content 5732 associated with the first content keyvalue 5712, encrypted content 5734 associated with the second contentkey value 5714, and encrypted content 5736 associated with the thirdcontent key value 5716. Thus, the unencrypted content 5732, theencrypted content 5734, and the encrypted content 5736 are the contentdata 5706, the content data 5708, and the content data 5710,respectively. It will be understood that the unencrypted content 5732,the encrypted content 5734, and the encrypted content 5736, and thecontent key values 5712, 5714, 5716 encoded in the CHIF file 5700 areunrecognizable and unusable unless decoded by the appropriate CHIFdecoder, as described in the various embodiments herein. In order todecrypt the encrypted content 5734 and the encrypted content 5736, thesecond content key value 5714 is stored in the key value store 5730 ofthe CHIF file 5726, along with encryption data 5722. Similarly, thethird content key value 5716 is stored in the key value store 5730 ofthe CHIF file 5726.

If different encryption data is used to encrypt the content data 5708and 5710, the different encryption data can be stored in the key valuestore 5730 in association with the respective content key values 5714and 5716. This thus creates an association made up of a content data keyvalue and an encryption data, used in decryption of content. In someembodiments, passphrases used to decrypt content would be known to userswho are meant to access the encrypted content, and users can provide apassphrase that is used with the encryption data associated with thecontent key value as indicated in the key value store 5730 to decryptcontent after the CHIF file 5726 is decoded by a decoder. In someembodiments, the encryption data can be an identifier forencryption/decryption keys stored on the user system 5702 or elsewhere,which can be retrieved using the identifier for decryption of theselectively encrypted content. It will be understood that any number ofunencrypted or encrypted data can be encoded in this manner in a CHIFfile.

When a CHIF file is decoded and encrypted content is encountered, a usermay see a message or other indicator that the content is encrypted andis only capable of being decrypted by authorized users. In someembodiments, an authentication process can then be initiated toestablish whether or not the user is authorized to access theencryption/decryption key for the content data. In some embodiments, anauthentication operation can be performed before decoding to establishthe rights of the user so that the user does not have to performauthentication to access data after decoding. For example, the CHIF file5726 in FIG. 57 can be decoded and, when a user attempts to access oneof the encrypted content 5734, 5736, the decoder uses a passphraseprovided by the user to attempt to decrypt the content. In someembodiments, the decoder can determine that decryption was unsuccessful,and provide a failure message to the user. In some embodiments, thedecoder decrypts the content using the provided passphrase, and, if thepassphrase is incorrect, the user is presented with useless orunrecognizable decrypted data. Different encrypted content may havedifferent access parameters. For example, the encrypted content 5734 mayonly be accessed by one or more certain individuals, accounts, ordevices, while the encrypted content 5736 may only be accessed by one ormore other individuals, accounts, or devices. If a user is authorized toaccess the encrypted content 5734 and/or the encrypted content 5736, thedecoder uses the data provided by the user with the encryption data 5722to successfully decrypt the second content key value 5714 and/or thethird content key value 5716. In some embodiments, secret keys can bestored in the user system 5702 or another system and retrieved by thedecoder for decryption of content. After successful decryption, theencrypted content 5734 and/or the encrypted content 5736 is presented tothe authorized user. Selective encryption of content data in this mannerallows for users of differing access levels to view certain portions ofdata stored in CHIF files, while restricting access to more sensitivedata, such as PII, stored in a CHIF file.

For example, if a CHIF file stores medical diagnostic or testinginformation, the CHIF file could have unencrypted portions pertaining toa diagnosis, and perhaps other useful information such as patientgeographic area or patient demographics like ethnicity, gender, or age(but not personal information such as name, address, birthdate, etc.),that can be used by a researcher to determine disease trends amongdiagnosed or tested patients. In addition to a diagnosis or testingresult such as a positive or negative test result, the unencryptedcontent can include testing types such as viral culture, real-timereverse transcription polymerase chain reaction, serology,hemagglutination inhibition, immunofluorescence, enzyme immunoassay,rapid diagnostic tests, or other testing types. The personal informationcan be encrypted and denied access except to authorized persons, such asa patient's primary care physician. In this manner, positive andnegative tests or diagnoses can be tracked by researchers to determinetrends among positive or negative cases, such as by testing type,ethnicity, gender, geographic area, or other factors, while alsoprotecting personal patient data or other sensitive data from access. Itwill be understood that each item of data, whether encrypted orunencrypted, such as diagnosis result, testing type, geographic area,name, address, birthdate, etc., can each be a separate content datastructure with an associated content data key value. It will beunderstood that, in various embodiments herein, the determination as towhich content is ultimately encrypted or left unencrypted is up to userscreating CHIF files, or coded parameters that encrypt certain dataroutinely as part of organizational policy.

FIG. 58 illustrates an example selective encryption process 5800 inaccordance with various embodiments of the present disclosure. Theprocess 5800 can be used with any of the systems and other processesdescribed herein. The process 5800 can be performed and executed by aprocessor such as the processor 6502 described herein, and/or by aserver such as the CHIF server 5202, or a processor of an edge devicesuch as edge device 5408. The process 5800 begins at block 5802. Atblock 5802, the processor, via a CHIF encoder, receives content data forencoding into a CHIF file. At block 5804, the processor displays anencryption prompt. For example, the processor can cause a display of auser device to display a prompt asking the user if any of the contentprovided for encoding into a CHIF file is to be selected for encryption.In some embodiments, a GUI can be provided allowing a user to seerepresentations of each content data item to be encoded, and to choosewhich of these to encrypt. In some embodiments, as each content item isprovided for encoding into the CHIF file, a user can be prompted as towhether that content item is also selected for encryption.

At decision block 5806, the processor determines if content is to beencrypted. If not, the process 5800 moves to block 5822 wherein theprocessor encodes the content data into a CHIF file and provides thecreated CHIF file. The process then ends at block 5824. If, at decisionblock 5806, the processor determines that there is content to beencrypted, the process 5800 moves to block 5808. At block 5808, theprocessor receives or retrieves the selected content for encryption. Atblock 5810, the processor retrieves content key values associated witheach of the content selections, as described for example with respect toFIG. 57 . At block 5812, the processor retrieves available encryptiondata, such as an initialization vector or hash function as describedwith respect to FIG. 57 . The processor can also receive a passphrase tobe used with the encryption data to derive a secret key to be used forencryption of the content data. In some embodiments, differentencryption data and/or passphrases can be used for each contentselection to be encrypted, or specific encryption data and/orpassphrases can be used for one or more content selections, whileanother can be used for one or more other content selections, dependinghow many differing levels of security the user wishes to place on thecontent data items. At block 5814, the content selections are encryptedusing the encryption data.

At block 5816, for each encrypted content selection, the processorassociates the encrypted data used to encrypt the content selection witha content key value for the encrypted content selection. As describedherein, in some embodiments multiple content keys can be associated withencryption data, or different content keys can be associated withdifferent encryption data. At block 5818, the processor encodes, in akey value store of a CHIF file, the associated encryption data andcontent key values for the encrypted content. At block 5820, theprocessor combines the encrypted content and the unencrypted content ina data portion of the CHIF file, completes encoding of the CHIF file,and provides the encoded CHIF file. In some embodiments, content keyvalues for unencrypted content and encrypted content are also encodedinto the data portion of the CHIF file. The process 5800 ends at block5822.

FIG. 59 illustrates an example selective decryption process 5900 inaccordance with various embodiments of the present disclosure. Theprocess 5900 can be used with any of the systems and other processesdescribed herein. The process 5900 can be performed and executed by aprocessor such as the processor 6502 described herein, and/or by aserver such as the CHIF server 5202, or a processor of an edge devicesuch as edge device 5408. The process 5900 begins at block 5902. Atblock 5902, the processor, via a CHIF decoder, receives and decodes theCHIF file, as described in the various embodiments herein. At decisionblock 5904, the processor determines if the decoded content includesencrypted content. In some embodiments, a flag or other indicator can beincluded in the metadata of the CHIF file indicating that the CHIF fileincludes encrypted content. If not, the process 5900 ends at block 5920.If so, the process 5900 moves to block 5906.

At block 5906, the processor receives a request to access the encryptedcontent. At block 5908, the processor determines the content dataassociated with the request stored in the content data portion of theCHIF file, and retrieves the content key value for the encrypted contentto which access is requested, as also described with respect to FIG. 57. In some embodiments, the content key value can be retrieved from thecontent data portion of the CHIF file. The processor can then locate thecontent key value stored in the key value store, and determineencryption data associated with the content key value. At block 5910,the processor requests authentication information from the user, such asa passphrase. In some embodiments, authentication information can beprovided by the user prior to block 5910, such that the processoralready has the authentication information when attempting to decryptrequested content. In some embodiments, a user may provide a secret keyto be used, such as choosing a secret key stored on the user's system,prompting the processor to attempt to use the provided secret key.

At decision block 5912, the processor determines if the user providedcorrect authentication information. For example, if the processorattempts to use a passphrase provided by a user along with encryptiondata stored in the key value store, such as an initialization vector orhash function, the processer, in some embodiments, can determine theattempt failed based on resulting data from attempting the decryptionusing the incorrect passphrase. In some embodiments, the processordecrypts the encrypted content using the provided passphrase, resultingin a failure to provide the original unencrypted content. If the correctauthentication is not provided, the process 5900 moves to block 5914. Atblock 5914, access is denied to the encrypted content. The process 5900then moves to decision block 5918, wherein the processor determineswhether access is requested to additional or other encrypted content.

If, at decision block 5912, the authentication information is correct,the process 5900 moves to block 5916. At block 5916, the processorretrieves or uses previously retrieved encryption data from the keyvalue store and, along with the provided authentication information,derives a secret key or decryption key and decrypts the requestedcontent for viewing by the authenticated user. The process 5900 thenmoves to decision block 5918. If, at decision block 5918, access isrequested to additional or other encrypted content, the process 5900moves back to block 5908 to retrieve the content key value for theadditional or other requested content to begin the decryption processfor that content. If, at decision block 5918, access is not requested toadditional or other encrypted content, the process 5900 ends at block5920.

FIG. 60 illustrates an example binder creation system 6000 in accordancewith various embodiments of the present disclosure. The system 6000includes a client device 6002. The client device 6002 includes a CHIFencoder 6004 configured to encode and combine data into a CHIF file, asdisclosed in the various embodiments herein. The CHIF encoder 6004 canalso create or update binder files that are container for multiple CHIFcontainers. A binder file can be useful for associating multiple CHIFfiles having common data, common ownership, or other factors that createan association between the CHIF files. For example, if an artist iscreating a new audio recording, a first recording can be recorded in afirst CHIF file, a second recording in a second CHIF file, and so on.Each version of the recording can be saved as each one is created in abinder file, essentially providing a container for multiple dataversions. As another example, initial patient diagnostic information canbe recorded in a first CHIF file, and further patient testing can berecorded in subsequent CHIF files, which are all stored upon creation ina binder file, such that all diagnostic information on a patient isstored together. In various embodiments herein, CHIF files can beimmutable containers that, once created, are not modified, such as notadding any additional content to the created CHIF file. Binderstherefore allow a degree of mutability for CHIF files, since subsequentCHIF files can be creates that include new information, updatedinformation, or new versions of content.

In the example illustrated in FIG. 60 , the CHIF encoder 6004 receives adata set 6006 to be encoded into a CHIF file. The CHIF encoder 6004encodes the data set 6006 into a first CHIF file 6008. Upon creation ofthe first CHIF file 6008, the first CHIF file 6008 can be stored in astorage 6010 as a standalone CHIF file, or a binder file 6012 can becreated to store the first CHIF file 6008. When creating the binder file6012, a CHIF manifest 6014 is created that provides information on thearrangement of CHIF files in the binder file 6012, such as an order ofthe stored CHIF files, CHIF file content or content types, content taginformation, versioning information, such as if the first CHIF file 6008is a first version, and subsequent CHIF files are subsequent versions,etc. In some embodiments, the CHIF manifest 6014 can provide graphicalrelationships between CHIF files, such as a graph database includingvarious relationship among the CHIF files stored in the binder 6012. Insome embodiments, the CHIF manifest 6014 can also be accessed to searchfor CHIF content using the various information concerning the CHIF filesin the CHIF manifest 6014. For example, if a medical professional wishesto search for all testing results for COVID-19, the manifest is accessedto look up which of the CHIF files in the binder 6012 are associatedwith COVID-19 test results.

As also illustrated in FIG. 60 , new data sets or updated data sets,such as data set 6016, can be encoded into CHIF files by the CHIFencoder 6004 and stored in the binder 6012. For example, data set 6016is encoded and stored as a second CHIF file 6018 in the binder 6012.When creating the second CHIF file 6018, the CHIF encoder accesses CHIFmanifest 6014 in the binder 6012 to update the manifest 6014 withinformation concerning the second CHIF file 6018. The CHIF manifest 6014is updated for each subsequently created CHIF file.

FIG. 61 illustrates an example binder creation process 6100 inaccordance with various embodiments of the present disclosure. Theprocess 6100 can be used with any of the systems and other processesdescribed herein. The process 6100 can be performed and executed by aprocessor such as the processor 6502 described herein, and/or by aserver such as the CHIF server 5202, or a processor of an edge devicesuch as edge device 5408. The process 6100 begins at block 6102. Atblock 6102, the processor encodes a first data set into a first CHIFfile, as described in the various embodiments herein. At block 6104, theprocessor stores the first CHIF file in a binder associated with one ormore characteristics, such as a person, facility, content identifier, orother characteristics. At block 6106, the processor creates a manifestfile including the storage location and various information concerningthe content of the first CHIF file.

At decision block 6108, the processor determines if additional data orupdated data associated with the one or more characteristics is providedto the processor via an encoder. If so, the process moves to block 6110.At block 6110, the processor encodes the additional or updated dataassociated with the one or more characteristics into a new CHIF file. Atblock 6112, the processor stores the new CHIF file in the binder. Atblock 6114, the processor updates the manifest file in the binder toinclude storage location information of the new CHIF file and variousother information concerning the content of the new CHIF file. Theprocess 6100 then moves to back to decision block 6108.

If, at decision block 6108, the processor determines there is noadditional or updated data, the process 6100 moves to block 6116. Atblock 6116, the processor receives a request to access binder filecontents. At block 6118, the processor retrieves the manifest file andoutputs data retrieved from the manifest file regarding a plurality ofCHIF files to a user. At block 6120, the processor receives a selectionof one or more CHIF files to access, and retrieves the selected CHIFfiles from the binder for decoding and viewing. In some embodimentsdescribed herein, content from the selected one or more CHIF files canbe presented simultaneously to the user. The process 6100 ends at block6122.

FIG. 62 illustrates an example secured CHIF container 6200 in accordancewith various embodiments of the present disclosure. The secured CHIFcontainer 6200 includes various control information in a metadataportion 6202 that enables the control of access to or decoding of theCHIF container 6200. It will be understood that the metadata portion6202 can also include other information as described in the variousembodiments herein, such as CHIF container decoding information. Acontent portion 6204 of the CHIF container 6200 includes encodedcontent, the access to which can be controlled, such as text data, imagedata, audio data, video data, or other data. For example, as illustratedin FIG. 62 , the content portion 6204 can include video data 6206, andother data 6208 such as image data like a thumbnail image or artassociated with the video data. The video data 6206 can be full moviefiles, commercials, video snippets such as sneak peaks or trailers,instructional videos, work-in-progress videos that can be disseminatedto others in a company to pick up working on the video, or other videocontent.

Access to the content portion 6204 can be managed using the controlinformation in the metadata portion 6202 to ensure that the contentportion 6204, such as the video data 6206, is not viewed or copiedwithout authorization. For example, a movie may be disseminated toreviewers and press ahead of an official release date for the movie.However, if a movie file is simply provided to the reviewers and press,the movie file becomes susceptible to copying and sharing with othersthe movie studio or other owner of the movie did not intend to view themovie ahead of the official release date. The CHIF container 6200, andotherwise as disclosed herein, can thus be used to secure the movie orother video content so that only authorized users can access thecontents.

For example, the metadata portion 6202 can include owner information6210 that identifies the owner of the content in content portion 6204,such as a movie studio or other owner. In some embodiments, the ownerinformation 6210 can be displayed to users that access the CHIFcontainer 6200, indicating that the content in content portion 6204 isowned or copyrighted by a third party, discouraging any unauthorizeddissemination. The metadata portion 6202 can also include a copy number6212. The copy number 6212 includes, in some embodiments, a numberindicating which numbered copy the CHIF container 6200 is of an originalCHIF file. For example, a plurality of unique CHIF containers, in anamount that a content owner wishes to provide to others, can be encodedto include the content in content portion 6204, with each created CHIFcontainer including an incremented number as the copy number 6212 in themetadata portion 6202. The copy number 6212 assists in tracking whichunique CHIF container is accessed, and such tracking can be utilize aviewed flag 6216. The viewed flag 6216 can be a binary, negative orpositive, 0 or 1, flag that indicates whether the CHIF container 6200has ever been decoded and viewed. In some embodiments, the viewed flag6216 can include a number of times the CHIF container 6200 has beendecoded and viewed.

The metadata portion 6202 can also include application restriction data6214 that indicates one or more specific applications that are allowedto decode and view the contents of the CHIF container 6200. For example,if a movie studio wishes to disseminate CHIF containers including amovie for advanced viewing, a specific CHIF decoding and viewingapplication can be provided to the reviewers. The applicationrestriction data 6214 can include an identification of the specificapplication, and include commands for rejecting any attempted decodingof the CHIF container 6200. In some embodiments, the applicationrestriction data 6214 can also include a list of applications that arenot allowed to access the CHIF container 6200.

The CHIF container 6200 can also include expiration data 6218 in themetadata portion 6202. The expiration data 6218 can include variousinformation concerning expiration of the CHIF container 6200, that is,parameters for when decoding of the CHIF container 6200 is no longerallowed. For example, the expiration data 6218 can include a certaindate of expiration, a parameter to expire the CHIF container 6200 whenthe viewed flag 6216 is positive, a parameter to expire the CHIFcontainer 6200 after a certain number of views, a parameter to expirethe CHIF file 6200 after an attempt to decode and view the CHIFcontainer 6200 using an application other than the applicationdesignated in application restriction data 6214, or other expirationparameters. To further control access to CHIF container 6200, the CHIFcontainer 6200 can be stored on a server that provides access to theCHIF container 6200 via the authorized application designated inapplication restriction data 6214. For example, a link to the copy ofthe remotely stored CHIF container 6200 can be provided to a user, whicheither launches the authorized application or is input into theauthorized application such that the authorized application remotelyretrieves the CHIF container 6200 for decoding and viewing. In someembodiments, the CHIF container 6200 can be decoded server-side, and thedecoded content is downloaded or streamed to the authorized application.

Upon the contents of the CHIF container 6200 being decoded and viewed,the CHIF container can be updated or recreated with the viewed flag 6216changed to positive, or a number of views updated. If expiration isdependent on viewing the contents of the CHIF container 6200 or a numberof views, subsequent attempts to retrieve, decode, or otherwise accessthe CHIF container 6200 can be denied. In some embodiments, CHIFcontainers stored on a device such as the CHIF server can be deletedupon expiration. In some embodiments, if an unauthorized decodingattempt of the CHIF container 6200 is detected, the CHIF container 6200can be remotely expired and even deleted.

FIG. 63 illustrates an example secured CHIF system 6300 in accordancewith various embodiments of the present disclosure. The system includesa CHIF server 6302 that can store CHIF containers for access by aplurality of client devices 6304 over a network 6306. The CHIF server6302 can be a single server or multiple servers that provide adistributed server network. The system 6300 further includes a contentowner device 6308. The content owner device 6308 can be a device used bya content creator that desires to provide content in one or more CHIFfiles to disseminate the content to certain users, such as the users ofthe plurality of client devices 6304. For example, the content ownerdevice 6308 can be an electronic device used by a movie studio to encodemovie files, trailers, or other video data into CHIF files fordissemination and viewing by others in a secured and restricted manner.As illustrated in FIG. 63 , the content owner device 6308 includes anencoder 6310, which is used to create a CHIF container 6312, such as aCHIF container including a movie or video file, along with control data,such as disclosed herein with respect to FIG. 62 . After creation of theCHIF container 6312, the content owner device 6308 can provide the CHIFcontainer 6312 to the CHIF server 6302 for storage of the CHIF container6312 on the CHIF server 6302. In some embodiments, the content ownerdevice 6308 can be the same devices as the CHIF server 6302, or a devicein the distributed server network.

In some embodiments, access to the contents of the CHIF container 6312by the plurality of client devices 6304 can be accomplished by providingaccess to the CHIF container 6312 to the plurality of client devices6304, such that the plurality of client devices 6304 each access thesame CHIF container, and the CHIF container 6312 can be updated orrecreated upon each access to have updated viewing information,expiration information, etc. In some embodiments, a predetermined numberof copies of the CHIF container 6312 can be created for access by theplurality of client devices 6304, with each of the plurality of clientdevices 6304 being provided access to one of the copies. In someembodiments, to reduce usage of storage space on the CHIF server 6302,copies of the CHIF container 6312 can be created on demand, such as inresponse to an access request from one of the plurality of clientdevices 6304. For example, as illustrated in FIG. 63 , copies of theCHIF container 6312 are created and stored on the CHIF server 6302,including a first copy 6314, a second copy 6316, and an nth copy 6318,access to each of which is requested by one of the plurality of clientdevices 6304. In some embodiments, the original CHIF container 6312 canhave initial or default control information encoded in the metadata, andthe copies each have different or updated control information, such as aspecific copy number, expiration date, or other control information.

In addition to encoding control information into the metadata of theCHIF container 6312, the CHIF server 6302 can also store CHIF controldata 6320 in association with the CHIF container 6312. The CHIF controldata 6320 includes various information that can be checked against thecontrol information of CHIF containers, to determine if access should bedenied, such as due to expiration of a CHIF container. For example, if aclient device of the plurality of client devices 6304 requests access tothe first copy 6314, the CHIF server 6302 can read the metadata in thefirst copy 6314 and compare the metadata to the CHIF control data 6320.For instance, the CHIF control data 6320 can indicate that access to theCHIF container copies should expire upon a single viewing or a certainnumber of viewings. If the viewed flag or a number of views indicated inthe metadata of the first copy 6314 meets this expiration parameter inthe CHIF control data 6320, access to the first copy 6314 is denied,and, in some embodiments, the first copy can be deleted so that the CHIFserver 6302 no longer has to process any further requests for access tothe first copy 6314. The CHIF control data 6320 can also include otherdata, such as authorized applications, expiration dates, or othercontrol information. In some embodiments, to further control access tothe CHIF files on the CHIF server 6302, each of the plurality of clientdevices 6304 must use a designated application 6322 to view requestedCHIF file content. In some embodiments, the designated application 6322submits the requests to access a CHIF file and remotely checks theaccess data for the CHIF file to determine if access is allowed. In someembodiments, the designated application 6322 can either download therequested CHIF file, or downloads or streams contents of the requestedCHIF file decoded server-side.

FIG. 64 illustrates an example secured CHIF container process 6400 inaccordance with various embodiments of the present disclosure. Theprocess 6400 can be used with any of the systems and other processesdescribed herein. The process 6400 can be performed and executed by aprocessor such as the processor 6502 described herein, and/or by aserver such as the CHIF server 5202 or 6302, or a processor of an edgedevice such as edge device 5408. The process 6400 begins at block 6402.

At block 6402, a CHIF container is encoded to include content data suchas video data or other data, and metadata including owner control data,such as described with respect to FIGS. 62 and 63 . At block 6404, theencoded CHIF container is stored at a predetermined storage location,such as CHIF server 6302 or another storage device. At block 6406, theprocessor creates a number of copies of the CHIF container for viewingand updates the metadata for each copy based on a copy number and anyother information such as custom expiration data. In some embodiments,the encoded CHIF file created and stored at block 6402 and 6404 may notbe copied, and access to the encoded CHIF file can be provided to aplurality of users.

At block 6408, the processor provides access information to at least oneof the copies of the CHIF container to a user or a user device. In someembodiments, the access information can include a link to the storedcopy of the CHIF container, or to the original CHIF container in someembodiments. This link can be used by a user to download a CHIF file, orremotely access a CHIF file, such as via a designated application thatat least partly controls access to CHIF files and that can download orstream CHIF file content. At block 6410, the processor detects anattempt to access the at least one of the copies of the CHIF containerby another device.

At decision block 6412, the processor determines if the access attemptis from a designated application. In some embodiments, the processor canlook up a designated application type from application restriction datain the metadata of the CHIF container, such as described with respect toFIG. 62 . In some embodiments, CHIF control data such as CHIF controldata 6320 can be consulted to determine the appropriate designatedapplication that is allowed to access the CHIF file and/or view contentsof the CHIF file. If, at decision block 6412, the processor determinesthat the application attempting to access or request the CHIF filecontents is not the designated application type, at block 6414, theprocessor denies access to the CHIF file. The process 6400 then ends atblock 6420. If, at decision block 6412, the processor determines theapplication attempting to access or request the CHIF file is thedesignated application type, the process 6400 moves to decision block6416.

At decision block 6416, the processor determines if the CHIF file hasexpired. For example, the metadata of the CHIF file or the CHIF controldata stored on the server can indicate various expiration parameters,such as also described with respect to FIGS. 62 and 63 . For example,the expiration parameters can include that the CHIF file expires after asingle viewing, and, upon examining a viewed flag of the CHIF file, theprocessor determines that the CHIF file is expired. The expirationparameters can be based on a number of views, an expiration date, anumber of copies, or other parameters. If, at decision block 6416, theprocessor determines the CHIF file is expired, the process 6400 moves toblock 6414 to deny access to the CHIF file. In some embodiments, theCHIF file can also be deleted upon determining the CHIF file is expired.The process 6400 then ends at block 6420. If, at decision block 6416,the processor determines the CHIF file is not expired, the process 6400moves to block 6418. At block 6418, the processor decodes the at leastone of the copies of the CHIF container and presents the content data tothe designated application. For example, if the content data includesvideo data, the video data can be decoded from the CHIF file anddownloaded or streamed to a client device running the designatedapplication. The content data can also be presented in the designatedapplication, to restrict the use of the content data to the designatedapplication. In some embodiments, the designated application can onlystore the content data temporarily and delete the content data as it ispresented, either periodically or at the end of presentation of thecontent data, so that the data cannot be accessed, copied, or otherwiseused by the user of the client devices besides viewing the content inthe designated application in response to the approvals at decisionblocks 6412 and 6414. In some embodiments, viewing of the content atblock 6418 can also trigger the processor to update CHIF control orexpiration parameters, such as updating a viewed flag or updating aviewed count. The process 6400 then ends at block 6420.

Referring to FIG. 65 , one embodiment of an electronic device or asystem device 6500 is illustrated. The system device 6500 is onepossible example of a device used by an end user, such as a mobiledevice, a device used in the systems described herein, such as theauthentication server 2502, CHIF servers such as servers 4208, 4408,5002, 5202, etc., or other devices. Embodiments include cellulartelephones (including smart phones), personal digital assistants (PDAs),netbooks, tablets, laptops, desktops, workstations, telepresenceconsoles, and any other computing device that can communicate withanother computing device using a wireless and/or wireline communicationlink. Such communications may be direct (e.g., via a peer-to-peernetwork, an ad hoc network, or using a direct connection), indirect,such as through a server or other proxy (e.g., in a client-servermodel), or may use a combination of direct and indirect communications.It is understood that the device may be implemented in many differentways and by many different types of systems, and may be customized asneeded to operate within a particular environment.

The system 6500 may include a processor or controller (e.g., a centralprocessing unit (“CPU”)) 6502, a memory unit 6504, an input/output(“I/O”) device 6506, and a network interface 6508. The components 6502,6504, 6506, and 6508 are interconnected by a transport system (e.g., abus) 6510. A power supply (PS) 6512 may provide power to components ofthe computer system 6500, such as the CPU 6502 and memory unit 6504, viaa power system 6514 (which is illustrated with the transport system 6510but may be different). It is understood that the system 6500 may bedifferently configured and that each of the listed components mayactually represent several different components. For example, the CPU6502 may actually represent a multi-processor or a distributedprocessing system; the memory unit 6504 may include different levels ofcache memory, main memory, hard disks, and remote storage locations; theI/O device 6506 may include monitors, keyboards, and the like; and thenetwork interface 6508 may include one or more network cards providingone or more wired and/or wireless connections to a network 6516.Therefore, a wide range of flexibility is anticipated in theconfiguration of the computer system 6500.

The system 6500 may use any operating system (or multiple operatingsystems), including various versions of operating systems provided byMicrosoft (such as WINDOWS), Apple (such as Mac OS X), UNIX, and LINUX,and may include operating systems specifically developed for handhelddevices, personal computers, servers, and embedded devices depending onthe use of the system 6500. The operating system, as well as otherinstructions, may be stored in the memory unit 6504 and executed by theprocessor 6502. For example, the memory unit 6504 may includeinstructions for performing some or all of the methods described herein.These instructions may reside within an application 6518. Theapplication 6518 may also include an application programming interface(API) 6520. The application 6518 may in some embodiments be the CHMcode, CHM encoder, CHM decoder, etc. In some embodiments, the API 6520may be an API for a CHM codec, CHM encoder, CHM decoder, etc., allowingfor API calls to be made in order to initiate CHM encoding and decodingoperations.

It will be understood that the CHM file, encoding and decodingoperations, and other processes described herein may include datacompression steps, encryption steps, or other processes to eitherdecrease the file size for transmission of the CHM file or provideadditional security. In various embodiments, CHM file and CHIF file canrefer to a same file type assembled using the systems and methodsdisclosed herein.

In an example embodiment, a method of a codec for encoding data streamsinto a combined file is provided. The method comprises accessing a firstfile having a first plurality of data bytes, accessing a second filehaving a second plurality of data bytes, combining the first file andthe second file to provide the combined file containing a header and abody, comprising the steps of in a first storing step, storing a blockof data bytes of a first byte block size, from the first plurality ofdata bytes, in the body of the combined file as a first file byte block,wherein a byte block size includes at least one or more bytes of data,in a second storing step, sequentially storing a block of data bytes ofa second byte block size, from the second plurality of data bytes, inthe body of the combined file as a second file byte block, repeating thefirst and second storing steps to sequentially store all of the databytes in the first file and the second file in the combined file, andstoring, in the header, information relating to the first byte blocksize and the second byte block size.

In one or more of the above examples, the first file is of a first fileformat and the second file is of a second file format.

In one or more of the above examples, the first file is an image fileand the second file is a non-image file.

In one or more of the above examples, the bytes are stored in sequentialand adjacent storage locations in the combined file.

In one or more of the above examples, the method further comprisestransferring the combined file to an application processing block todecode the combined file into the first and second files by the steps ofreading the header to determine the byte block size of each of the firstand second files, sequentially accessing a number of bytes associatedwith the byte block size of the first file and a number of bytesassociated with the byte block size of the second file, and creating thefirst and second files with the accessed bytes.

In one or more of the above examples, the byte block size of the firstfile and the byte block size of the second file are calculated inaccordance with a ratio that is a number of bytes of the first file to anumber of bytes of the second file.

In one or more of the above examples, the step of calculating the byteblock size for each of the first file and the second file includesdetermining which of the first file and the second file includes agreater number of bytes, dividing the number of bytes of the first fileor the second file that includes the greater number of bytes by theother one of the first file or the second file to produce a result,determining if the result includes a remainder and, if so, rounding theresult up to an integer that is a next integer up from the result, andsetting the byte block size of the first file or the second file thatincludes the greater number of bytes to be equal to the integer.

In one or more of the above examples, if a total number of data blocksin either of the first and second data files causes, after all previousbytes in the first or second file are written, there to be a number ofbytes left in either the first or second file that is, for the firstfile, less than the first byte block size, or, for the second file, lessthan the second byte block size, the method further comprises storing apartial byte block in the combined file, wherein the partial byte blockis associated with one of the first or second files and wherein thepartial byte block includes a number of data bytes less than the byteblock size for the first or second file that is associated with thepartial byte block.

In one or more of the above examples, the step of calculating the byteblock size for each of the first file and the second file furtherincludes setting the byte block size of the second file to one byte anddetermining if a speed multiplier is to be set and, if so, setting thespeed multiplier, wherein the speed multiplier is a value to manipulateby the byte block size of the first file and the byte block size of thesecond file in order to set the byte block size of the first file to theresult of the value multiplied by the byte block size of the first fileand to set the byte block size of the second file to the result of thevalue multiplied by the byte block size of the second file.

In another example embodiment, a method of a codec for decoding a datastream of a combined file into separate data streams is provided. Themethod comprises analyzing header data included in the combined file,calculating a byte block size for each of a first data stream and asecond data stream, wherein a byte block is a number of bytes of datawithin a file, reading a first file byte block included in the combinedfile, wherein the first file byte block includes a number of bytes inthe combined file corresponding to the byte block size calculated for afirst data stream, writing the first file byte block to a first file,reading a second file byte block included in the combined file, whereinthe second file byte block includes a number of bytes in the combinedfile corresponding to the byte block size calculated for a second datastream, and writing the second file byte block to a second file.

In one or more of the above examples, the first file is of a first filetype and the second file is of a second file type.

In one or more of the above examples, the first file is an image fileand the second file is a non-image file.

In one or more of the above examples, the method further comprisesdetermining if each byte included within the combined file has been readfrom the combined file and written to one of the first file or thesecond file and repeating, upon a determination that each of the bytesincluded within the combined file have not been written to one of thefirst file or the second file, the reading, writing, and determiningsteps.

In one or more of the above examples, calculating the byte block sizefor each of a first data stream and a second data stream includesreading byte block size data in the header data, wherein the byte blocksize data includes byte block sizes used during creation of the combinedfile.

In one or more of the above examples, a system for encoding data streamsinto a combined file and decoding the combined file into separate datastreams is provided. The system comprises a network interface coupled toa processor and a memory to the processor, the processor configured toaccess a first file having a first plurality of data bytes, access asecond file having a second plurality of data bytes, combine the firstfile and the second file to provide the combined file containing aheader and a body, wherein, during combining, the processor is furtherconfigured to in a first storing step, store a block of data bytes of afirst byte block size, from the first plurality of data bytes, in thebody of the combined file as a first file byte block, wherein a byteblock size includes at least one or more bytes of data, in a secondstoring step, sequentially store a block of data bytes of a second byteblock size, from the second plurality of data bytes, in the body of thecombined file as a second file byte block, repeat the first and secondstoring steps to sequentially store all of the data bytes in the firstfile and the second file in the combined file, and store, in the header,information relating to the first byte block size and the second byteblock size.

In one or more of the above examples, the first file is of a first fileformat and the second file is of a second file format.

In one or more of the above examples, the first file is an image fileand the second file is a non-image file.

In one or more of the above examples, the step of calculating the byteblock size for each of the first file and the second file includesdetermining which of the first file and the second file includes agreater number of bytes, dividing the number of bytes of the first fileor the second file that includes the greater number of bytes by theother one of the first file or the second file to produce a result,determining if the result includes a remainder and, if so, rounding theresult up to an integer that is a next integer up from the result, andsetting the byte block size of the first file or the second file thatincludes the greater number of bytes to be equal to the integer.

In one or more of the above examples, the processor is furtherconfigured to analyze header data included in the combined file,calculate the byte block size for each of the first file and the secondfile based on byte block size data included in the header data, record acombined file byte block included in the combined file, wherein thecombined file byte block includes a number of bytes in the combined filecorresponding to the byte block size calculated for the first file, copythe combined file byte block to a third file, record a next combinedfile byte block included in the combined file, wherein the next combinedfile byte block includes a number of bytes in the combined filecorresponding to the byte block size calculated for the second file, andcopy the next combined file byte block to a fourth file.

In one or more of the above examples, the processor is furtherconfigured to ascertain whether each byte included within the combinedfile has been read from the combined file and written to one of thethird file or the fourth file, and repeat, upon a determination thateach of the bytes included within the combined file have not beenwritten to one of the third file or the fourth file, all of the record,copy, and ascertain steps.

In another example embodiment, a method of blending an image file with anon-image file is provided. The method may begin with accessing bothimage and non-image data streams or files by the application programwhich implements the technology set forth by this disclosure. Onceaccessed, the application may read the data information of both imageand non-image streams or files, and based on the chm format, it maycreate and write the metadata bytes into the beginning header section ofa chm format data stream or file. Next, the application may read onechunk of data bytes from each of the image and non-image data streams orfiles, and write the two chunks of data bytes into the chm format streamor file. And the application may continue and repeat reading one chunkof data bytes from two data streams and writing the two chunks of databytes into the chm format data stream or file until it reaches the endof the two image and non-image data streams or files. The process ofthis method is called “chm encoding.”

In another example embodiment, a method of separating the chm formatdata stream or file and returning back the image stream or file and thenon-image stream or file is provided. The method may begin withaccessing the chm format data stream or file which is generated by theblending method above with the application program. Once accessed, theapplication may read and retrieve the metadata information from a headersection of the stream or file. Next, based on protocol, the applicationmay read two chunks of bytes from the chm format data stream or file,and it may write one chunk of bytes into the image stream or file, andanother chunk of bytes into the non-image stream or file. And theapplication may continue and repeat reading the next two chunks of bytesfrom the chm format data stream, and writing each chunk of the bytesinto their own data streams or files until it reaches the end of the chmformat data stream or file, and it returns the image and non-image datastreams or files back to their original states. The process of thismethod is called “chm decoding.”

In another example embodiment, a method for encoding and decodingdisparate content includes receiving, at a storage location, a combinedfile created by an encoder, wherein the combined file includes aplurality of data of one or more content types. The method furtherincludes receiving, from a client device, a request for retrieval of thecombined file. The method further includes determining a compatibilityof the client device to decode and view content of the combined file.The method further includes transmitting, based on the determination,the combined file to a decoder, wherein the decoder separates thecombined file into the plurality of data of the one or more contenttypes.

In another example embodiment, an electronic device includes a memory, anetwork interface, and at least one processor coupled to the memory andthe network interface. The at least one processor is configured toreceive a combined file created by an encoder, wherein the combined fileincludes a plurality of data of one or more content types. The at leastone processor is further configured to receive, from a client device, arequest for retrieval of the combined file. The at least one processoris further configured to determine a compatibility of the client deviceto decode and view content of the combined file. The at least oneprocessor is further configured to transmit, via the network interface,and based on the determination, the combined file to a decoder, whereinthe decoder separates the combined file into the plurality of data ofthe one or more content types.

In another example embodiment, an electronic device includes a memory, anetwork interface, and at least one processor coupled to the memory andthe network interface. The at least one processor is configured toreceive, from a client device, a request for decryption and decoding ofa combined file, wherein the combined file includes metadata andplurality of data of one or more content types, wherein the metadataincludes a universally unique identifier (UUID), and wherein thecombined file is encrypted. The at least one processor is furtherconfigured to decrypt the combined file. The at least one processor isfurther configured to determine, based on the UUID, that decoding of thecombined file is allowed. The at least one processor is furtherconfigured to decode, based on the determination, the combined file,wherein the decoding separates the combined file into the plurality ofdata of the one or more content types. The at least one processor isfurther configured to transmit, via the network interface, the pluralityof data of the one or more content types to the client device.

In another example embodiment, an electronic device includes a memoryand at least one processor coupled to the memory. The at least oneprocessor is configured to receive a combined file created by anencoder, wherein the combined file includes metadata and an imageassociated with a plurality of data of one or more content types. The atleast one processor is further configured to feed at least one of theplurality of data into at least one artificial intelligence model. Theat least one processor is further configured to receive one or moreoutputs from the at least one artificial intelligence model. The atleast one processor is further configured to create an enriched combinedfile by modifying the metadata of the combined file based on at least aportion of the one or more outputs from the at least one artificialintelligence model, The at least one processor is further configured toperform on the enriched combined file at least one of: analytics,indexing, or object recognition.

In another example embodiment, a method of an electronic device forselective encryption of content to be encoded into a container includesreceiving, by a processor of the electronic device, a plurality ofcontent items for encoding into a container, wherein each one of theplurality of content items is associated with a content key value,receiving, by the processor, one or more content selections from amongthe plurality of content items, retrieving, by the processor, encryptiondata for use in encrypting the one or more content selections,encrypting, by the processor, at least one content selection of the oneor more content selections using an encryption key derived using theencryption data, for the encrypted at least one content selection,associating, by the processor, a corresponding content key value of theencrypted at least one content selection with the encryption data,encoding, by the processor, the associated corresponding content keyvalue of the encrypted at least one content selection and the encryptiondata into the container, and encoding, by the processor, the encryptedat least one content selection and remaining content items of theplurality of content items into the container.

In one or more of the above examples, the encrypted at least one contentselection is encoded into the container as encrypted content and theremaining content items of the plurality of content items are encodedinto the container as unencrypted content.

In one or more of the above examples, the method further includesencrypting, by the processor, another content selection of the one ormore content selections using another encryption key derived using theencryption data, for the encrypted other content selection, associating,by the processor, another corresponding content key value of theencrypted other content selection with the encryption data, encoding, bythe processor, the associated other corresponding content key value ofthe encrypted at least one content selection, and encoding, by theprocessor, the encrypted other content selection into the container.

In one or more of the above examples, the method further includesretrieving, by the processor, second encryption data, encrypting, by theprocessor, another content selection of the one or more contentselections using another encryption key derived using the secondencryption data, for the encrypted other content selection, associating,by the processor, another corresponding content key value of theencrypted other content selection with the second encryption data,encoding, by the processor, the associated other corresponding contentkey value of the encrypted at least one content selection and the secondencryption data into the container, and encoding, by the processor, theencrypted other content selection into the container.

In one or more of the above examples, the encrypted at least one contentselection and the other content selection are encoded into the containeras encrypted content and the remaining content items of the plurality ofcontent items are encoded into the container as unencrypted content.

In one or more of the above examples, the processor uses the encryptiondata to derive the encryption key from a passphrase provided to theprocessor.

In one or more of the above examples, the method further includesdecoding the container into decoded content, receiving a request toaccess the encrypted at least one content selection, identifying, fromthe decoded content, the associated corresponding content key value forthe at least one content selection, retrieving the encryption dataassociated with the identified associated corresponding content keyvalue, and decrypting the encrypted at least one content selection.

In one or more of the above examples, decrypting the encrypted at leastone content selection includes receiving a passphrase by the processor,deriving a decryption key using the passphrase and the encryption data,and decrypting the encrypted at least one content selection using thederived decryption key.

In one or more of the above examples, the at least one content selectionincludes text data.

In one or more of the above examples, the associated correspondingcontent key value of the encrypted at least one content selection andthe encryption data are encoded into a key value storage portion of thecontainer separate from a content data portion of the container in whichthe encrypted at least one content selection is encoded.

In another example embodiment, an electronic device includes a memory,and at least one processor coupled to the memory. The at least oneprocessor is configured to receive a plurality of content items forencoding into a container, wherein each one of the plurality of contentitems is associated with a content key value, receive one or morecontent selections from among the plurality of content items, retrieveencryption data for use in encrypting the one or more contentselections, encrypt at least one content selection of the one or morecontent selections using an encryption key derived using the encryptiondata, for the encrypted at least one content selection, associate acorresponding content key value of the encrypted at least one contentselection with the encryption data, encode the associated correspondingcontent key value of the encrypted at least one content selection andthe encryption data into the container, and encode the encrypted atleast one content selection and remaining content items of the pluralityof content items into the container.

In one or more of the above examples, the encrypted at least one contentselection is encoded into the container as encrypted content and theremaining content items of the plurality of content items are encodedinto the container as unencrypted content.

In one or more of the above examples, the at least one processor isfurther configured to encrypt another content selection of the one ormore content selections using another encryption key derived using theencryption data, for the encrypted other content selection, associateanother corresponding content key value of the encrypted other contentselection with the encryption data, encode the associated othercorresponding content key value of the encrypted at least one contentselection, and encode the encrypted other content selection into thecontainer.

In one or more of the above examples, the at least one processor isfurther configured to retrieve second encryption data, encrypt anothercontent selection of the one or more content selections using anotherencryption key derived using the second encryption data, for theencrypted other content selection, associate another correspondingcontent key value of the encrypted other content selection with thesecond encryption data, encode the associated other correspondingcontent key value of the encrypted at least one content selection andthe second encryption data into the container, and encode the encryptedother content selection into the container.

In one or more of the above examples, the encrypted at least one contentselection and the other content selection are encoded into the containeras encrypted content and the remaining content items of the plurality ofcontent items are encoded into the container as unencrypted content.

In one or more of the above examples, the processor uses the encryptiondata to derive the encryption key from a passphrase provided to theprocessor.

In one or more of the above examples, the at least one processor isfurther configured to decode the container into decoded content, receivea request to access the encrypted at least one content selection,identify, from the decoded content, the associated corresponding contentkey value for the at least one content selection, retrieve theencryption data associated with the identified associated correspondingcontent key value, and decrypt the encrypted at least one contentselection.

In one or more of the above examples, to decrypt the encrypted at leastone content selection, the at least one processor is further configuredto receive a passphrase, derive a decryption key using the passphraseand the encryption data, and decrypt the encrypted at least one contentselection using the derived decryption key.

In one or more of the above examples, the at least one content selectionincludes text data.

In one or more of the above examples, the associated correspondingcontent key value of the encrypted at least one content selection andthe encryption data are encoded into a key value storage portion of thecontainer separate from a content data portion of the container in whichthe encrypted at least one content selection is encoded.

In another example embodiment, a method for encoding and decodingdisparate content includes receiving, at a storage location, a combinedfile created by an encoder, wherein the combined file includes aplurality of data of one or more content types, receiving, from a clientdevice, a request for retrieval of the combined file, determining acompatibility of the client device to decode and view content of thecombined file, and transmitting, based on the determination, thecombined file to a decoder, wherein the decoder separates the combinedfile into the plurality of data of the one or more content types.

In one or more of the above examples, transmitting, based on thedetermination, the combined file to the decoder includes identifyingthat the compatibility of the client device includes that the clientdevice comprises one or more applications for decoding and presentingthe plurality of data of the one or more content types simultaneously,and transmitting the combined file to the client device.

In one or more of the above examples, transmitting, based on thedetermination, the combined file to the decoder includes identifyingthat the compatibility of the client device includes that the clientdevice does not include one or more applications for decoding andpresenting the plurality of data of the one or more content typessimultaneously, and transmitting the combined file to the client device,wherein the decoder is associated with the client device and wherein theseparated plurality of data of the one or more content types is eachstored separately in association with the client device.

In one or more of the above examples, the method further includesreceiving from the client device a request to use a viewing portal toview the plurality of data of the one or more content types, decoding,by the decoder, the combined file, wherein the decoder is stored inassociation with the storage location, providing the plurality of dataof the one or more content types to a viewer application, andtransmitting, via the viewer application, a simultaneous output of theplurality of data of the one or more content types to the client device.

In one or more of the above examples, the method further includesreceiving the combined file in a locked state, wherein the locked stateis a state in which the combined file cannot be edited, and wherein thecombined file is transitioned into the locked state upon appending anauthorized signature to the combined file.

In one or more of the above examples, the method further includesstoring, in association with the combined file, an index, wherein theindex includes searchable information regarding the combined file,receiving a search request from the client device, wherein the searchrequest include one or more search parameters, retrieving one or morecombined files based on the one or more search parameters, and based onone or more indexes associated with the one or more combined files,creating a binder file, wherein the binder file includes informationregarding the one or more combined files, and transmitting the binderfile to the client device.

In one or more of the above examples, the method further includesdetermining that a total size of the one or more combined files exceedsa threshold, and creating the binder file such that the binder fileincludes retrieval links to the one or more combined files.

In one or more of the above examples, the method further includesdetecting execution of a decoder script, determining an identity of auser executing the decoder script, and blocking decoding of the combinedfile based on the determination of the identity of the user executingthe decoder script.

In one or more of the above examples, the encoder creates the combinedfile by accessing a first file including a first plurality of databytes, accessing a second file including a second plurality of databytes, and combining the first file and the second file to provide thecombined file including only one header and one body. Combining thefirst file and the second file includes the steps of in a first storingstep, sequentially storing a first file byte block of a first byte blocksize in the body of the combined file, wherein the first file byte blockincludes one or more bytes of data from the first plurality of databytes of the first file, in a second storing step, sequentially storinga second file byte block of a second byte block size in the body of thecombined file, wherein the second file byte block includes one or morebytes of data from the second plurality of data bytes of the secondfile, repeating the first and second storing steps to sequentially storethe first plurality of data bytes of the first file and the secondplurality of data bytes of the second file in the body of the combinedfile, and storing, in the header of the combined file, a number of bytesof the first file and the second file, the first byte block size, andthe second byte block size, wherein the first byte block size and thesecond byte block size are determined according to a file sizerelationship between the first file and the second file for use in adecoding process of the combined file.

In one or more of the above examples, the first plurality of data bytesand the second plurality of data bytes are both stored in the one bodyof the combined file in association with only the one header separatefrom the body, and wherein no byte block from the first file is storedadjacent another byte block from the first file, and no byte block fromthe second file is stored adjacent another byte block from the secondfile.

In one or more of the above examples, the decoder separates the combinedfile into the plurality of data of the one or more content types byanalyzing header data included in a header of the combined file, whereinthe combined file includes only one header and includes both data from afirst data stream of a first original file and data from a second datastream of a second original file in one body of the combined file inassociation with only the one header separate from the body, readingfrom the header data a number of bytes of the first original file andthe second original file, and a byte block size for each of a first datastream and a second data stream, wherein a byte block includes one ormore bytes of data within a file, and wherein the byte block size foreach of the first data stream and the second data stream are determinedaccording to a file size relationship between the first original fileand the second original file, reading a first file byte block includedin the combined file, wherein the first file byte block includes anumber of bytes in the combined file corresponding to the byte blocksize for the first data stream read from the header of the combinedfile, writing the first file byte block to a first file, reading asecond file byte block included in the combined file, wherein the secondfile byte block includes a number of bytes in the combined filecorresponding to the byte block size for the second data stream readfrom the header of the combined file, and writing the second file byteblock to a second file.

In another example embodiment, an electronic device includes a memory, anetwork interface, and at least one processor coupled to the memory andthe network interface. The at least one processor is configured toreceive a combined file created by an encoder, wherein the combined fileincludes a plurality of data of one or more content types, receive, froma client device, a request for retrieval of the combined file, determinea compatibility of the client device to decode and view content of thecombined file, and transmit, via the network interface, and based on thedetermination, the combined file to a decoder, wherein the decoderseparates the combined file into the plurality of data of the one ormore content types.

In one or more of the above examples, to transmit, based on thedetermination, the combined file to the decoder, the at least oneprocessor is further configured to identify that the compatibility ofthe client device includes that the client device comprises one or moreapplications for decoding and presenting the plurality of data of theone or more content types simultaneously, and transmit the combined fileto the client device.

In one or more of the above examples, the at least one processor isfurther configured to receive from the client device a request to use aviewing portal to view the plurality of data of the one or more contenttypes, decode, by the decoder, the combined file, wherein the decoder isstored in association with a storage location, provide the plurality ofdata of the one or more content types to a viewer application, andtransmit, via the viewer application, a simultaneous output of theplurality of data of the one or more content types to the client device.

In one or more of the above examples, the at least one processor isfurther configured to receive the combined file in a locked state,wherein the locked state is a state in which the combined file cannot beedited, and wherein the combined file is transitioned into the lockedstate upon appending an authorized signature to the combined file.

In one or more of the above examples, the at least one processor isfurther configured to store, in association with the combined file, anindex, wherein the index includes searchable information regarding thecombined file, receive a search request from the client device, whereinthe search request include one or more search parameters, retrieve oneor more combined files based on the one or more search parameters, andbased on one or more indexes associated with the one or more combinedfiles, create a binder file, wherein the binder file includesinformation regarding the one or more combined files, and transmit thebinder file to the client device.

In one or more of the above examples, the at least one processor isfurther configured to determine that a total size of the one or morecombined files exceeds a threshold, and create the binder file such thatthe binder file includes retrieval links to the one or more combinedfiles.

In one or more of the above examples, the at least one processor isfurther configured to detect execution of a decoder script, determine anidentity of a user executing the decoder script, and block decoding ofthe combined file based on the determination of the identity of the userexecuting the decoder script.

In one or more of the above examples, to create the combined file, theencoder is configured to access a first file including a first pluralityof data bytes, access a second file including a second plurality of databytes, and combine the first file and the second file to provide thecombined file including only one header and one body, wherein, duringcombining, the encoder is further configured to in a first storing step,sequentially store a first file byte block of a first byte block size inthe body of the combined file, wherein the first file byte blockincludes one or more bytes of data from the first plurality of databytes of the first file, in a second storing step, sequentially store asecond file byte block of a second byte block size in the body of thecombined file, wherein the second file byte block includes one or morebytes of data from the second plurality of data bytes of the secondfile, repeat the first and second storing steps to sequentially storethe first plurality of data bytes of the first file and the secondplurality of data bytes of the second file in the body of the combinedfile, and store, in the header of the combined file, a number of bytesof the first file and the second file, the first byte block size, andthe second byte block size, wherein the first byte block size and thesecond byte block size are determined according to a file sizerelationship between the first file and the second file for use in adecoding process of the combined file.

In one or more of the above examples, the decoder is configured to, toseparate the combined file into the plurality of data of the one or morecontent types analyze header data included in a header of the combinedfile, wherein the combined file includes only one header and includesboth data from a first data stream of a first original file and datafrom a second data stream of a second original file in one body of thecombined file in association with only the one header separate from thebody, read from the header data a number of bytes of the first originalfile and the second original file, and a byte block size for each of afirst data stream and a second data stream, wherein a byte blockincludes one or more bytes of data within a file, and wherein the byteblock size for each of the first data stream and the second data streamare determined according to a file size relationship between the firstoriginal file and the second original file, read a first file byte blockincluded in the combined file, wherein the first file byte blockincludes a number of bytes in the combined file corresponding to thebyte block size for the first data stream read from the header of thecombined file, write the first file byte block to a first file, read asecond file byte block included in the combined file, wherein the secondfile byte block includes a number of bytes in the combined filecorresponding to the byte block size for the second data stream readfrom the header of the combined file, and write the second file byteblock to a second file.

In another example embodiment, an electronic device includes a memory, anetwork interface, and at least one processor coupled to the memory andthe network interface. The at least one processor is configured toreceive, from a client device, a request for decryption and decoding ofa combined file, wherein the combined file includes metadata andplurality of data of one or more content types, wherein the metadataincludes a universally unique identifier (UUID), and wherein thecombined file is encrypted, decrypt the combined file, determine, basedon the UUID, that decoding of the combined file is allowed, decode,based on the determination, the combined file, wherein the decodingseparates the combined file into the plurality of data of the one ormore content types, and transmit, via the network interface, theplurality of data of the one or more content types to the client device.

In another example embodiment, an electronic device includes a memory,and at least one processor coupled to the memory. The at least oneprocessor is configured to receive a combined file created by anencoder, wherein the combined file includes metadata and an imageassociated with a plurality of data of one or more content types, feedat least one of the plurality of data into at least one artificialintelligence model, receive one or more outputs from the at least oneartificial intelligence model, create an enriched combined file bymodifying the metadata of the combined file based on at least a portionof the one or more outputs from the at least one artificial intelligencemodel, and perform on the enriched combined file at least one of:analytics, indexing, or object recognition.

It should be understood that the drawings and detailed descriptionherein are to be regarded in an illustrative rather than a restrictivemanner, and are not intended to be limiting to the particular forms andexamples disclosed. On the contrary, included are any furthermodifications, changes, rearrangements, substitutions, alternatives,design choices, and embodiments apparent to those of ordinary skill inthe art, without departing from the spirit and scope hereof, as definedby the following claims. Thus, it is intended that the following claimsbe interpreted to embrace all such further modifications, changes,rearrangements, substitutions, alternatives, design choices, andembodiments.

What is claimed is:
 1. A method of an electronic device for selectiveencryption of content to be encoded into a container, the methodcomprising: receiving, by a processor of the electronic device, aplurality of content items for encoding into a container, wherein eachone of the plurality of content items is associated with a content keyvalue; receiving, by the processor, one or more content selections fromamong the plurality of content items; retrieving, by the processor,encryption data for use in encrypting the one or more contentselections; encrypting, by the processor, at least one content selectionof the one or more content selections using an encryption key derivedusing the encryption data; for the encrypted at least one contentselection, associating, by the processor, a corresponding content keyvalue of the encrypted at least one content selection with theencryption data; encoding, by the processor, the associatedcorresponding content key value of the encrypted at least one contentselection and the encryption data into the container; and encoding, bythe processor, the encrypted at least one content selection andremaining content items of the plurality of content items into thecontainer.